from fastai import * # Quick access to most common functionality
from fastai.text import * # Quick access to NLP functionality
An example of creating a language model and then transfering to a classifier.
path = untar_data(URLs.IMDB_SAMPLE)
path
PosixPath('/home/ubuntu/fastai/examples/data/imdb_sample')
Open and view the independent and dependent variables:
df = pd.read_csv(path/'texts.csv', header=None)
df.head()
| 0 | 1 | 2 | |
|---|---|---|---|
| 0 | label | text | is_valid |
| 1 | negative | Un-bleeping-believable! Meg Ryan doesn't even ... | False |
| 2 | positive | This is a extremely well-made film. The acting... | False |
| 3 | negative | Every once in a long while a movie will come a... | False |
| 4 | positive | Name just says it all. I watched this movie wi... | False |
Create a DataBunch for each of the language model and the classifier:
data_lm = TextLMDataBunch.from_csv(path, 'texts.csv')
data_clas = TextClasDataBunch.from_csv(path, 'texts.csv', vocab=data_lm.train_ds.vocab, bs=42)
We'll fine-tune the language model. fast.ai has a pre-trained English model available that we can download, we jsut have to specify it like this:
moms = (0.8,0.7)
learn = language_model_learner(data_lm, pretrained_model=URLs.WT103_1)
learn.unfreeze()
learn.fit_one_cycle(4, slice(1e-2), moms=moms)
Total time: 01:36 epoch train_loss valid_loss accuracy 1 4.977893 4.187432 0.249392 (00:24) 2 4.619538 4.074967 0.253530 (00:23) 3 4.328201 4.020354 0.261265 (00:24) 4 4.082543 4.019140 0.259830 (00:24)
Save our language model's encoder:
learn.save_encoder('enc')
Fine tune it to create a classifier:
learn = text_classifier_learner(data_clas)
learn.load_encoder('enc')
learn.freeze()
learn.fit_one_cycle(4, moms=moms)
Total time: 01:38 epoch train_loss valid_loss accuracy 1 0.657817 0.621307 0.736318 (00:23) 2 0.614628 0.503447 0.810945 (00:25) 3 0.579545 0.493127 0.766169 (00:23) 4 0.588436 0.482557 0.791045 (00:25)
learn.unfreeze()
learn.fit_one_cycle(8, slice(1e-5,1e-3), moms=moms)
Total time: 07:28 epoch train_loss valid_loss accuracy 1 0.551981 0.475971 0.800995 (00:58) 2 0.574055 0.495363 0.756219 (00:59) 3 0.551674 0.459942 0.796020 (00:55) 4 0.532213 0.444801 0.791045 (00:50) 5 0.519754 0.427174 0.810945 (00:56) 6 0.509370 0.424626 0.830846 (00:50) 7 0.502223 0.415141 0.845771 (00:56) 8 0.518687 0.411042 0.835821 (01:00)