Julien Salinas, Founder & CTO of NLP Cloud – Interview Sequence


Julien Salinas is the Founder & CTO of NLP Cloud. The NLP Cloud platform serves excessive efficiency production-ready NLP fashions based mostly on spaCy and HuggingFace transformers, for a number of use instances together with NER, sentiment evaluation, textual content classification, summarization, query answering, textual content technology, translation, language detection, grammar and spelling correction, intent classification, and semantic similarity.

What initially bought you interested by laptop science?

I began programming in… enterprise faculty! I do know it sounds shocking. Really, I shortly realized that enterprise itself was boring and that I’d be shortly restricted if I didn’t have the technical expertise to realize my initiatives.

The primary mission on the time was a small web site for my music trainer, then one other one for my household, then I began studying Python… and so forth and so forth. Now I’ve been a Python/Go developer and DevOps for 15 years.

Might you share the genesis story behind NLP Cloud?

It began 2 years in the past after I realized that, as a developer, it was sort of exhausting to correctly deploy machine studying fashions into manufacturing.

I used to be amazed by the progress made by frameworks like Hugging Face Transformers and spaCy, and I used to be in a position to leverage very superior NLP fashions in my initiatives. However utilizing these fashions in manufacturing was one other beast and, surprisingly, I couldn’t discover any fascinating No-Ops cloud in the marketplace for NLP.

So, I made a decision to begin my very own platform for NLP fashions deployment. In a short time we had nice buyer feedbacks and we added many options based mostly on these feedbacks (pre-trained fashions, fine-tuning, playground…).

The NLP Cloud platform helps the GPT-3 open-source different GPT-J. What’s GPT-J particularly?

GPT-J has been launched by a workforce of researchers known as EleutherAI in June this 12 months. They imagine that GPT-3 must be an open-source mannequin, like its predecessors (GPT and GPT-2). They declare that, even when we should always all be involved about potential misuse of highly effective AI fashions like GPT, it’s not cause to not make these mannequin open-source. Fairly the other: they imagine that if AI fashions stay open-source, it’s the easiest way for the neighborhood to grasp how these fashions are working beneath the hood, after which guarantee that these fashions don’t behave the flawed approach (misogyny, racism, …).

GPT-J is a direct equal to GPT-3 Curie as each are skilled on roughly 6 billion parameters.

Each can virtually be used interchangeably.

Why is GPT-J a superior different to GPT-3?

GPT-3 belongs to Microsoft and the one approach for folks to make use of it’s to undergo the official GPT-3 API.

However this API may be very costly, and intensely restrictive: it’s worthwhile to request entry to the API and, even when your utility will get accepted, your entry could be shut down anytime in the event that they think about that your online business mannequin doesn’t adjust to their tips. For examples, you possibly can’t generate “open-ended” textual content (lengthy textual content made up of a number of paragraphs) because it’s towards their coverage.

There is no such thing as a such restrictions with GPT-J because it’s open-source and anybody can set up it and use it.

What have been a number of the technical challenges with integrating GPT-J on NLP Cloud?

GPT-J is advanced to put in due to its excessive useful resource consumption (RAM, CPU, GPU…). It really works and not using a GPU however it’s so sluggish that it’s very impractical.

In the long run, the {hardware} wanted to run GPT-J may be very costly so, in an effort to decrease the prices, we needed to work on many implementation particulars.

Additionally, in an effort to guarantee high-availability of GPT-J on NLP Cloud and make it suited to manufacturing, we needed to work on redundancy and failover methods for GPT-J that may be fairly difficult.

Might you focus on a number of the pre-trained AI fashions which are supplied?

We’re doing our greatest to pick out one of the best pre-trained AI mannequin per use case.

For textual content summarization, one of the best one – in our opinion – is Fb’s Bart Massive CNN that provides superb outcomes however that may be fairly sluggish and not using a GPU.

For textual content classification, we applied Fb’s Bart Massive MNLI (for English classification) and Joe Davison’s XLM Roberta Massive XLNI (for non-English languages). Each are quick and really correct.

For query answering, we use Deepset’s Roberta Base Squad 2. It’s quick and correct however for extra superior query answering you may need to use GPT-J.

And lots of others!

What are a number of the finest use instances for NLP Cloud?

The use instances that appear to be used essentially the most are textual content summarization, textual content classification, and textual content technology with GPT-J for product description generations, paraphrase, article technology…

However the use instances we are able to see amongst our clients are extraordinarily various, and it’s fairly spectacular to witness so many nice concepts developing!

Is there the rest that you simply want to share about NLP Cloud?

It appears to us that AI for textual content understanding and textual content technology is lastly used “for actual” in precise merchandise or inner workflow, by increasingly corporations.

That is nice to see that NLP just isn’t solely a pure analysis discipline anymore, however that there are actual enterprise use instances that may leverage NLP.

At NLP Cloud we’ll preserve doing our greatest to make it simple for anybody to check and use NLP in manufacturing.

Thanks for the good interview, readers who want to study extra ought to go to NLP Cloud.