Language AI — the Age of Disruption

When fruits are not just fruits anymore

Computer scientists often joke that “life was so much simpler when apples and blackberries were just fruits" . What’s truly remarkable is that today’s computers can also get the joke. They can recognize which apple we’re talking about (the fruit or the brand) by using proximity and similarity to establish context. For example, they are looking at other words in the text like pie , tree , tasty or earnings , computer , Tim Cook . And with every processed sentence, machines become better at it. The way they achieve this is through language models based on transformers.

These transformers are not the humanlike robots we see on films, but machine learning models that are revolutionizing how we analyze information at scale. This has implications for how we work, learn, manage our money, or look after our health. Transformers are incredibly complex models that digest vast amounts of inputs all at once — we're talking trillions of parameters.

They can also identify relationships between words and make deductions. This gives machines a “sense” of context and the ability to extract meaning. It is this pivot from words to meaning that makes Language AI so disruptive.

From disappointment to excitement

Language AI is at an exciting turning point, but it hasn’t been a smooth ride.

About 70 years ago, scientists started exploring whether machines could imitate human activities, including “ thinking ”. In 1950, Alan Turing first ran the now famous “ Turing test ”, in which someone asks questions via a terminal to both a human and a machine. Based on the answers provided, they needed to guess which terminal was operated by a computer.

8 years later, John Mcarthy released the programming language LISP (acronym for list processing), the foundation of language AI. For the first time programmers had access to flexibility and organization.

The pivot towards training

70 years ago, it was thought that machines could learn anything and everything without a proper education.

Since then, we realized not only that language models needed specialized training, but that they would perform only as well as the data they were trained on. There is no one know-it-all, universal AI the way scientists initially imagined it; instead, it comes in the form of specialized models designed for very detailed tasks. So researchers are now creating hyperfocused training sets that are meant to help solve specific, real world problems.

Let’s say we want Language AI machines to help us forecast inflation more accurately, by counting and reading job listings from employers across all industries — and in real time. We first need to teach computers about workforce-related concepts such as companies, industries, occupations, and skill and the many ways they can be interlinked. Once we've done this, we can start asking computers for answers, such as — are companies or sectors growing faster or signaling a hiring freeze? That information, put in the context of how job growth impacts inflation, can make our economic predictions more accurate and timely.

This is a huge endeavor. It involves creating ontologies, taxonomies and quilting together knowledge graphs, which are interlinked descriptions of objects, events or abstract concepts. All this is essential in adding context and depth to machine learning techniques, but takes a lot of resources and can get out of control. So the next frontier is scalability. And we're seeing a lot of progress in speeding up training, making deployments of large language models simple, transparent and cost effective.

So far, no AI has been able to indisputably pass the Turing test and be mistaken for a human. That’s fine, because I think machines should help us evolve, not outsmart us or be like us. The ultimate goal is to empower everyone (governments, NGO, companies and individuals) to leverage the power of AI using big data. Because data driven decisions are wiser and more sustainable.

Here’s where we are today — in a place of incredible promise and momentum, with machine learning driving a paradigm shift in information technology. Just like fitting in a CPU inside a personal computer led to the democratization of computing, Language AI will disrupt the way we think, brainstorm, and communicate with others.

To get there, we need to continue building solid foundations — hyper focused on using big data to train models together with knowledge graphs of facts and human values. If the wrong foundations are laid, it is possible that language AI could lead to unpredictable consequences and stall progress once again.