How to build a machine learning model when dataset is small?

#1

Interesting answer on building a model when you have small dataset

https://www.quora.com/What-are-some-machine-learning-algorithms-that-can-learn-well-even-with-a-small-data-set/answer/Xavier-Amatriain?srid=nlkH

#2
#3

It really depends on the dataset as well. For example if you have a dataset which is small in the sense where classes are imbalanced then you can use oversampling techniques to generate datapoints for that particular class.This is a way to generate synthetic points in the data. One of such techniques is Synthetic Minority Oversampling Technique.

#4

Intuitively, you can:

  1. Make more data
  2. Use a model appropriate for little data

Making data - This means using techniques, such as Augmentation or MLE to create more samples using the same data.

Models that can work on little data - Decision Trees work by naturally slicing the dataset. Decision trees can provide good performance even on small datasets, as opposed to deep neural networks which can’t. For some ML algorithms, small data is enough data.