Training Data Quotes

Once your computer is pretending to be a neural net, you get it to be able to do a particular task by just showing it a whole lot of examples.

Algorithms learn by being fed certain images, often chosen by engineers, and the system builds a model of the world based on those images. If a system is trained on photos of people who are overwhelmingly white, it will have a harder time recognizing nonwhite faces.

Want to train a machine translation system? Train it on a gazillion pairs of sentences of parallel corpora, and that creates a lot of breakthrough results. Increasingly, I'm seeing results on small data where you want to try to take in results even if you have 1,000 images.

The challenge for what everybody's seeing in deep learning - the software richness is really quite high. In training, you have to wait days and weeks before it comes back to tell you whether your model works or not. And in the beginning, they all don't work.

Except for Wikipedia, maybe, no independent website has provided as much raw training data for as many AI firms, authorized or not, as Reddit. As a corpus for machines trying to sound or reason like people, it’s immensely valuable: pre-organized, pre-moderated, cleaned and sorted by the input of millions of volunteers and users, and written, unlike so much else on the web, without SEO, traffic, or advertisers in mind. Likewise, its relationship with OpenAI runs deeper than the deal announced in early 2024 through which Reddit licensed data to OpenAI for training and to bring Reddit content directly into ChatGPT.

AIs are only as good as the data they are trained on. And while many of the tech giants working on AI, like Google and Facebook, have open-sourced some of their algorithms, they hold back most of their data.

I think the first wave of deep learning progress was mainly big companies with a ton of data training very large neural networks, right? So if you want to build a speech recognition system, train it on 100,000 hours of data.

Humans are unbelievably data efficient. You don't have to drive 1 million miles to drive a car, but the way we teach a self-driving car is have it drive a million miles.

We should always be suspicious when machine-learning systems are described as free from bias if it's been trained on human-generated data. Our biases are built into that training data.

AIs are only as good as the data they are trained on.

All you need is lots and lots of data and lots of information about what the right answer is, and you'll be able to train a big neural net to do what you want.

The neural network is this kind of technology that is not an algorithm, it is a network that has weights on it, and you can adjust the weights so that it learns. You teach it through trials.