Amin Ahmad is the CTO and co-founder of Vectara, a company which enables firms worldwide to train and deploy AI models at scale. Amin was awarded a Spotlight Trailblazer award in AI.
Spryte Spotlight: AI Specialist Simone Faricelli
An Interview with Stip AI's Simone Faricelli
Highlights
What you really need to be a part of Stip is to be passionate about artificial intelligence and cutting edge technologies. We are always looking for new solutions, researching and developing new models. We are not stopping at just one good solution.
About
Get to know Spotlight leaders. In our interviews we delve deep into how they think and what they’re building.
View the full interview
Watch the full interview with AI specialist Simone Faricelli
I think the world all over is going a little bit crazy with AI and so a lot of people are interested and concerned, and there's a lot of miscommunication going on. I think a lot of people are maybe reading a little too much into what the future looks like, some people maybe are not realizing what the future looks like, or maybe in some cases, the future is already here and we just don't know about it. So I'm going to start very big picture, and it may be a difficult question to answer, but how do you feel about AI? You're an AI specialist. This is your focus in your career for a while now. How do you feel about AI in general and humankind? Are you concerned? Are you excited? Is this a positive net, a positive something to be worried about? How do you feel?
I may be a bit on one side because I am an AI specialist, of course, but what I think is that we have to be excited about this new AI, every day new stuff is coming out, because I think that AI is not replacing people. AI is a tool at the service of humanity. We have to use it. We have to be aware that it might be something really dangerous, but in the hands of bad people. So if we make good use of AI, we can make ourselves better, our work better. We can really speed up processes. We can make things really easy. So I think we don't have to think about a catastrophic use of artificial intelligence like in the Terminator, Matrix, or whatever film with dystopian visions. I'm very proud of what I do. This artificial intelligence is changing the world and in a better way. And I hope it will do this even better next time.
OK. So you're excited, right? But overall, what would you say is the emotion that you feel right now? Is it more excitement than fear?
Absolutely. It's exciting. I have no fear because I know that at the moment, AI cannot do anything. It's very focused on specific tasks, so we don't have to be too afraid of artificial intelligence.
All right. Well, a lot of people are probably excited about that answer. You're an AI specialist, a lot of people right now who are looking for jobs, looking for better jobs, would love to know, how do you become an AI specialist?
Oh, that's a really interesting question. You know, I was an informatic engineer. And at a certain point of my life, I had to choose between cybersecurity and artificial intelligence. But I have this kind of... I really love how people think, people's brains. I really love how to understand how people behave in general. And I thought that it would be nice to join these two passions of mine, which is... IT and people's brains, people's behavior. And this has joined into artificial intelligence. Because of course, artificial intelligence is taken from people's brains in general. So we turn how people behave into artificial intelligence models. And this is really an amazing thing to me.
Got it. So, the original impetus for the original models were based on the human neuron model. We've taken from biology and we've built these models based on that. Through doing this, is there something we've learned about human cognition or the way we learn or the way our brain works?
Yeah, absolutely. I mean, the basic stuff of artificial intelligence is based on neurons. So activation function is basically how the electrons move through our brains. Biology and artificial intelligence are really born together in this sense. But has AI had any repercussions on people that study the brain in general and neurology? Have we been able to take that information back? Has that enlightened us into the way humans function a little bit or not yet? I don't actually know. We know for sure that AI helped a lot with prosthetic arms and stuff like that. But I don't know if we can translate something from artificial intelligence to a human being. We know for sure that we can do the opposite, turning the behavior of talking, seeing and hearing into an artificial intelligence model then task. But I don't know if we can actually do the opposite.
Yeah, maybe not there yet. I guess it's kind of fitting because I think the product that you're working on at Stip, it's not just AI related to maybe tagging videos, or it's not a pure computing function. You're really bringing AI into the fold of human process, right? How do you improve human processes and human engagement? How do you improve customer service? And so I think that idea you gave about the origins of your interest into AI, I think is fitting. I interview a lot of CTOs, a lot of people in technical fields, and I always find that the original motivations for people joining the field really guide the type of companies that they end up working with. I think that's definitely the case here. What kind of attracted you to Stip and how did you get involved with that product?
So Stip is born from an idea or maybe a situation. We all have been in the situation of contacting a big company, the customer care of this big company for whatever issue we have. and we really hated having this low functioning bot answering us and not solving our issue. This happens not only because of the malfunctioning but also because the human being wants to feel the warmth of another person. A human being wants to speak with one of his own, so the main reason behind Stip is not trying to replace humans with artificial intelligence, but providing an artificial intelligence tool to customer service operators of big companies to make their work easier. We wanted to speed up the process, help the operator. We want human and artificial intelligence to work together to achieve one common goal.
Okay, so you started with the idea of “let's improve customer service”, right? And then what was your first approach towards building something that worked? Because I think a lot of AI companies or a lot of companies overall have this problem right now where the typical product development cycle for software is usually based on an idea. And then you build the software based on that idea. And it seems to me that in AI, it's a little bit different. I think a lot of times you maybe have to start with the data and you can't really do much without the data. Was that the case? Did you experience data problems when you began or how did you manage that, bridge that gap?
Yes, we’ve tried many models in these five years of work. At the beginning, our models were very data hungry. We needed a lot of data from each customer to build a functioning artificial intelligence model. But moving through the years and collecting general data, we were able to build a model that can actually do tasks, perform very well, even without any kind of data. And thanks also to these new upcoming generative models like OpenAI, the new Elon Musk XAI that released GROK, and other models like that. Now we are able to exploit this model, or better, the idea behind this model, the architecture that lies behind these models. And we are able to help a company even without adding a single piece of data from them. Obviously, with the data from the company, we can build the customer-specific models that are visibly better, but we can help even without any data.
So it seems like the large language models have sort of taken it up a notch, right? By providing a base on which to improve. When did you guys make that switch internally or when were you able to leverage these large language models internally and get that into your product?
Okay, internally we have a good research and development field, which is always finding new solutions. So we were trying to use these generative models for two years, and now we’ve been able to develop our own model and make it better, make it good for the customer care domain specifically. We have a lot of domains, we have automotive finance, we have clothing, food and beverage. We have a lot of general data that we can use to make a really good general purpose model.
I think what you're describing seems enormously expensive to me because the people that are necessary to develop these types of models are very expensive, right? They're all highly educated and highly sought after. I think it's one of the things that's super surprising to us about your company is the fact that you're able to do all of this while... not being, you know, endlessly funded, right? I guess you guys are funded by your clients. I don't know if you're revenue positive yet, but you're at least building with a client base and basing that off of the revenue rather than having enormous funding from VCs or anything like that. How did you guys manage that?
Yes, we received big funding at the beginning of STIP, but right now we have some good Italian companies that rely on us and they are spending a lot of money on STIP. They are grateful for the product that we provide them. And yes, also the people have to be trained, not only the model, in order to generate a good product. But it's something that we really like, so we do it without any kind of trouble. We really enjoy doing it.
How did you guys go about building your team internally? I think everybody wants to build a world leading edge AI team. How did you guys go about it?
I think that what you really need to be a part of Stip is to be passionate about artificial intelligence and cutting edge technologies. Because as I told you, we are always looking for new solutions, researching and developing new models. We are not stopping at one good solution. We are always moving towards newer and newer solutions and technologies.
How do you structure your team internally? Because you've got active clients to service, and then you've got this research and development group. Is it a separate group completely? Or how do you structure your teams?
Actually, we try to do everything. Everyone can choose to do research and development or choose to provide the solution to a client. But it's basically the same thing, because as new clients come in, we try to provide them new solutions. So we research while serving the client at the same time. We also do this separately, but every client is a challenge for us that needs a new solution.
So these new solutions, these new models, I don't know how much you can describe it. How does it come about, the idea generation of that? Is it through investigating the data? Is it through talking to clients about their needs? How do you guys come up with the next feature or next model to build?
Basically, everyone in Stip and outside Stip can have a good idea to build a new model. Some good ideas came up from our CEO, who is not really knowledgeable about AI, but he came up with really good ideas, so we tried to make it work. And sometimes it goes good and sometimes it doesn't, but we provided really good models by starting with these ideas from different people, outside of the AI field.
Got it. And then in terms of the development lifecycle, from idea to finished model that you can present to a new customer or to an existing customer, is there a common pattern in terms of timeline or does it vary?
Yes, what's common is our need to evaluate the idea as soon as possible. Because if we cannot do anything, we’ll know for sure at the beginning of the project. But if we think that the idea could be good and we could provide a good solution, then we move on to the development phase, the study and development phase, and then we provide the production solution.
And in the last couple of years, what does that timeline look like for a new feature? Is it six months? Is it 12 months? Is it 24 months?
It depends on the feature. I think that for new models, we spend around three months developing a new model for a customer. And if we need to develop the same model for a new customer, that will take a week or two.
Okay, cool. That seems pretty compressed compared to a lot of companies out there. So tell me a little bit about the challenges with everybody who's involved with customer service and operations. You know, you always have to cobble multiple systems together. There's never one system that your team is working with. How did you guys develop your product to work with multiple systems? And what are some of the challenges that you faced in developing these very special models that let you do what you do?
I think that the biggest challenge at the moment is in regards to the hardware resources that we need to train and use for the new generational model. For instance, Elon Musk's company XAI just released its generational mixer of an expert model, which has 314 billion parameters, which is really a lot because it's double the amount of the GPT-3 from OpenAI. But what this means is that the model is not runnable on a single GPU. Meaning that a normal person will never be able to run this model on their own. So the problem, the challenge, is having these really heavy computational resources available for us to train and to test the model. This is really expensive also because once we have found the model, once we have trained it, we need this model to be ready. It needs to be on a computer, on a server, on a really powerful resource available at any given time. And this is a really expensive task. And also at the moment, we have not, not only us, but like AWS, Google, other providers don’t have these kinds of resources available because a lot of people, a lot of companies are investing in AI. So they are all trying to have these resources by their home. This is the biggest challenge at the moment.
So computational resources and I guess both access and costs are a problem. And how do you guys solve that? Were you cloud-based since the beginning? Or do you guys have internal GPUs and server farms that you guys have relied on? How do you guys solve that issue?
As much as we can, we are using cloud-based resources. But as I told you, they are not available all the time. So we're trying to build a local solution, a big computer with a very GPU intensive workload and stuff like that.
Yeah, I mean, that's a big risk considering the scale of NVIDIA's development into new chips, new GPUs, and a lot of chip companies out there. It's also a challenge in terms of privacy, security of the data. So it's really a big task right now. Do you see a light at the end of the tunnel for that computational problem? Or do you think this is going to be a driving problem for the foreseeable future?
I actually don't know how the world will solve this problem because it's not just us, it's all over the world. I mean, models are getting open sourced more and more over time. So everyone wants the models and they want it locally because they don’t want their data to be sent to Google, AWS, or whoever it is. So I don't know. The world is looking for resources more and more. And I don't know how we are going to solve this problem. I think we're building new computers, new GPUs, more powerful than previous ones. But also the models are getting bigger and bigger, as I told you.
And I think we have been seeing the sort of problem with allocation of resources, right? I think a lot of smaller companies have a hard time getting access because the giants out there have the money to spend and they're monopolizing the resources a lot of times. Have you guys seen that? Have you been impacted through, I guess what you're describing is exactly that, the fact that both resource and cost is hitting into your overall ability to develop, right?
Yes, we struggle to allocate new resources from cloud providers because big companies manage to allocate so many resources and we’re not always able to get our little space, our little room.
And I think we've seen that even impact hiring where the researchers and the people who are looking for a company to get in bed with, and they are actually sometimes even demanding to see like, call me when you've got 100,000 GPUs that I can play with, right? I think it impacts the level of people that you can even hire unless you have those resources.
Yes, I think for the R&D departments of big companies, they are not using cloud resources, but local resources that make them able to develop really quickly. But for the production world, this is not happening because every client needs their machine, every client needs their model. It's not always easy to allocate these resources and we cannot have a new computer for every client, let's say.
I guess all of that cost has got to be passed on to the customer somehow. What's your experience with customer understanding of AI products and AI costs? It all seems fairly complicated to have to project the number of tokens or the overall pricing that you're going to need? How do you interface with your customers to explain the scale and the cost involved?
Yes the idea is that every product has a cost and for a generative AI coming from Stip the cost is not so cheap. But companies are understanding the cost of these new models, also because I think since OpenAI is open to everyone and you can see the prices, you can calculate the amount of money you will spend on it. So I think they are actually understanding the cost of every model we generate for them.
With operations, I think a lot of times you have to figure out how much time am I going to save? How many resources am I going to save? What's the logic on the operational side for a customer to switch over to Stip? How do they have to approach that logic?
It always depends on the kind of tool they choose, because we have a set of AI models starting from a very, very basic one. I mean, we can also have just a model that can understand if a message needs to be carried or not. So that is an easy solution that does not imply any great expense. But as the model becomes bigger, the solution becomes bigger, the client has to wait more time for the solution to be built. And obviously it needs more money to be spent on it.
On your site you've got a bunch of different models. Can you maybe go into a little bit of the types of models, the AI models that you guys use?
Yes. As I told you, there is a very basic one, which is called by us, Caring, No Caring, which just tells us if a message needs to be curled by an operator or not. And if it does, what is the category that this message belongs to? Obviously, the category depends on the company because every company, every domain has its own categories. We always try to make a solution that is client specific. They are not the only models. Obviously, we also have an identity recognition model, which helps the operator to very quickly provide the solution to the ticket that comes in. In fact, this model is like highlighting the piece of the messages that are more important for the operator, like the trouble, the issue, and what's the cause of this issue, which is very important for a customer care operator. Then we also have a semantic search engine, which helps the operator to find, in their history, the closest cases that have already been solved by another operator. Now we also have a generative model that can answer users directly. Sometimes it is based on the history and sometimes it can generate an answer based on the domain. Like I told you, we don't really need the data from all the operators, from all the clients to build a working solution. We can also use the data we already have of that specific domain to build a good solution. That's basically the kind of models we have.
And then you actually train each one of those models on specific clients' data for each customer?
We have the general model trained on the domain and then we can also build a new model trained on the specific client domain which will obviously always perform better than the general purpose model, but not all clients are able to do that as they are not able to send us their data due to GDPR restrictions.
I think data privacy and anonymization is an issue that you guys are getting into. I'll just maybe ask one final question. What can we look forward to in 2024? Are there any special features or developments that you guys are excited about?
What we are trying to do with our current goal for the next few years is spread the idea that better communication between customers and companies is actually doable. And this is thanks to Stip. We want to make this communication better. We want to improve the operators' work by helping them with artificial intelligence. And we want to do this internationally, not only in Italy. So we are trying to build a team which is international and also multi-ethnic. This is our main goal for the next few years. But also, let's say, technically speaking, what we want, what we really want is to build a solution that is completely automatic. We don't want to... We want a complete automated pipeline with the goal of generating a set of AI tools trained on the client-specific domain, that can be used by our clients without the need of time- and effort-consuming processes from our developers. We just want the client to fit these models with their data. So upload the data, upload their Excel files, choose the tools they need, and they are done. The solution is already out for them.
Browse Articles
Macro Fiore was awarded the Spryte Spotlight Visionaries award for his outstanding contribution and groundbreaking product NetAI which focuses on networking and security to accelerate TELECOM operations and make cellular networks more efficient.
We spoke with modl.ai's CEO, Christoffer Holmgård, about his background, the history of modl.ai, and the future of AI in game testing.
Diffblue received the Spryte Spotlight Company to watch award for their groundbreaking product - the only fully-autonomous AI-powered Java and Kotlin unit test writing solution, generates reliable unit tests at scale, locally and in CI environments.
Krishna Tammana was awarded the Spryte Spotlight Company to Watch award for his outstanding contribution and groundbreaking product Gupshup which builds AI-powered chatbots that truly engage customers and help organisations level up their customer service game.