I'm the co-founder as well, yeah. Co-founder.

Yeah, for me, AI is a technological tool. Like, humanity has developed many of these tools since the beginning, like knives and steam engines and so on. So yeah, the use of these tools will necessarily lead to problems. They can be misused, but they will also bring about many good things.

I would rather say it's on the nuclear energy scale. It is good, but of course you can also misuse it or it can lead to fatal catastrophes.

It's both here I said every tool can be used for good as well as for bad things, so how um you know when did you actually get into you know at Diffblue you guys are you know you're dealing directly with code and you're using ai to help people code or maintain code when did you actually get started what what sort of give me give me the origin story of DiffBlue. So we started almost eight years ago. And we started from my background in software verification with the goal to make sure that software does what it is supposed to do. And testing is obviously one method for doing that. So what we wanted to do is to use technology to make testing much easier, in particular for developers that spend a lot of time writing tests. And this is something that can be automated by putting some intelligence in tools. This is how we started out. Our tagline of the company was pretty soon, or AI for code, because this is what we provide as a company.

I think what the public perceived at the bend of the eye pretty much is a revolution. But in reality, the development has been much more incremental over many decades. What was really revolutionary, in my opinion, was how The new interface that OpenAI came up with was making it easier for everyone to interact with it. And this has dramatically increased the visibility of the state of the art where we actually are in AI research. And I would say the power of the user interface, this was really revolutionary.

Much less, because you, as a developer, have certain expectations. Yeah. And, you see it can sometimes do great things, but sometimes it just spits out garbage and you just discard it and move on. So it's a mixed deck. Sometimes it helps, sometimes it doesn't.

Yeah, so our system has... has a functionality to actually make sure that we always produce the correct result. So software is a bit special because we have the software artifacts that we can execute. So we have actual access to the ground truth and we can make sure that the outputs that our AI produces are actually correct. Which is very different from many other applications where you yeah, everything is fluffy. You can't really tell what is right or wrong that easily, but the software is much easier. You can execute and you actually see what is going on. You see what the behavior is. And this is what we exploit in our system to make sure that we always give the correct results.

Sorry, I didn't get the question.

Yes, yeah. So we provide software developers with a tool that automatically writes unit tests. And we do this on two levels. First of all, it's integrated in the CI system of a project. So it runs autonomously on the code base and provides the unit tests. many advantages. First of all, it removes the burden from developers of actually doing anything. Secondly, for the managers, it's great to achieve or enforce a certain standard of unit testing across many code bases that their teams might be involved with and they might be at different levels. And so they can use our tool to get a certain amount of unit test coverage across their entire estate. And this is, of course, a big advantage. In particular, when they have projects that are at very different maturity stages, where legacy projects that have no testing at all and other projects where the developers maybe have a certain culture of not writing tests or being too lazy and so on. So this is really a tool that enables managers to get grips on their code base and states. On the other hand, we have also a plugin that developers can use from their IDE to write tests for the code they're currently working on.

Yeah, certainly, yes. Yeah, yeah. So, you know, I think then in your case, you're covering the, you know, code bases that are with a group of developers who's not handling unit testing very well. And that covers that automatically. Can you share some insight into sort of how that could even work? Like what's the secret sauce within AI that would enable that to be possible? So the key ingredient is that it needs to work autonomously. So you need to write, it's not like when you use Copilot that you get some suggestion and then it doesn't quite work, you have to fix it up, you have to make it compile and so on. So to really produce code autonomously, it must work out of box without any human intervention. And the system that we, the secret sauce behind that is that we, we use reinforcement learning. So we don't purely rely on some pre-trained models to produce something, but we actually learn as we proceed what the correct code will be. So we use that to optimize simultaneously to achieve the right code coverage, but, and also the, so that the code is idiomatic and looks like a human write them. And at the same time, make the code correct. So we do these things all at once, and the result is that we can produce code that actually compiles and runs and behaves as expected.

Exactly, yes. Got it.

No, this doesn't create 100% coverage. It's always the best effort. It often depends on how the code base is actually written. So it's quite easy to write code that is not really testable, not by a human hand, not let alone by a machine. So there is... So you won't get 100% coverage. It depends a bit on how well your code is written so that you get good coverage. But what we see in our customers is that on really bad projects, we get 30% coverage. On really good written projects, we get easily 80% and so on without any human intervention.

Yeah, so testability is usually a question of good architectural practices. So the usual story of high cohesion, low coupling, If a code is written that well, then it's also easy to test. And the same is true for a machine. If it has to write tests for code that is a plate of spaghetti, then this is really hard or almost impossible.

Exactly, yes. Got it. Okay. Well, that's good.

Exactly. And approaches like TDD, they aim at the same. You're using TDD because it enables you to produce better code that is better architectured, that has less coupling, and so on.

It depends. pretty much on the culture and how you approach a project. So to use TDD, you really need to know all the requirements upfront, which is a bit at odds with certain HR practices where you rather have a feedback process to incrementally discover the requirements and build your process more incrementally. So TDD has certain problems in this respect. But what we generally recommend is that you write the tests for the basic requirements that you know upfront using a TDD approach. And this is a very good practice because these tests guide you during the implementation. And then there are many, many corner cases that you still need to write tests for. And for these cases, automated test writing solution is perfectly suited.

Yeah. So we're focusing on unit testing because this is where most people are lacking and there is most to be gained from. Other types of tests are certainly an option. People are usually quite good at writing end-to-end tests because this is what they're most familiar with, although there is a lot to be automated there. There's quite some potential. For us as Deepflow, the current strategy is to stay at unit testing because this is what there is still a lot to be done in helping companies to adapt unit testing, which is necessary also to benefit from continuous integration. Because if you have no unit test to run, then your continuous integration is not very useful because you don't get fast feedback.

I'd certainly know, you know, you guys are focusing on Java, so I would get, I always get finance. Those guys need a lot of help in maintaining older code bases. Any other industries other than finance, which are sort of responding well to your product? Yeah, finance is certainly one of our sweet spots. There is certain interest from Tilker and Farmer and other companies that have quite substantial Java code bases.

It It was a business strategy because Java is, uh, is by far the largest language for backend systems. And the business decision was to focus on backend systems. So we said, okay, backend system is Java. So let's start with that.

There's certainly differences in the languages from a low-level technical perspective.

In principle, yes. There are no fundamental challenges. It's essentially just engineering.

There's certainly the controversial topic around copyright of the training data that still causes quite some anxiety, I would say. So we at Deepflow, we don't have this problem because we... All our training data is very carefully curated, but other companies that use models where the training data is, yeah, is less creative have that problem to um argue that yeah there are no ip implications if you use this tool and generate code with it and so on so there is so customers are very um interested in these questions and want to know that they're on the safe side

This is one way that is, of course, for many customers a no-go because they don't want any of their data to leave the premises. But also the other way around, if you train on some GPL code, and then that gets infiltrated in your code base, then of course this is undesirable.

So we start with a base model that is fully under our control, and then we use reinforcement learning on the customer code. to learn from the customer.

It's usually a matter of a few weeks I would say depends of course on the number of projects that they want to onboard but usually for yeah full integration into CI and training of the developers can usually be done in two to three weeks. This is mostly DevOps setup time and training. The computer itself doesn't take weeks.

Of course, the level of coverage is always a question. But most customers understand the enormous benefit that they get from the tool. So any increase by 20, 30% is massive. Because it will take the developers years or decades to do this work manually. So they... So this is relatively easy to understand that you won't get 100% coverage, but even if you just get a few percent, you have saved a lot of money and time. The questions around data security IP protection is very important. So nothing, no code should leave the premises. And all the code that is generated by the tool needs to be owned by the company, obviously, by the customer.

You guys are installing in their cloud systems or on their hardware? Yes. So our solution is on-prem. So you can install it. Usually it is installed in the CI system, so it's extremely lightweight because we use a small model and reinforcement learning, so it runs within a CI worker. It doesn't need a lot of compute. It doesn't call out to some cloud systems. It's quite easy to deploy.

Yeah. So we have a, um, a community edition. which provides only the plugin that you can use in your ID right now. So you can just download it now if you want. And that one runs locally on your laptop. So it's lightweight enough to just run on your laptop. We currently don't have an individual license. We are working on that because we have lots of users of the Community Edition who are not part of large enterprises who can afford an enterprise license, but still want to use it for their projects on a continuous basis without being restricted by the restrictions of the Community Edition. So we are working on that, yeah. Right

So the community edition is open for anybody to use. And this is one of our main entry points for new customers. Usually some developers download the tool, try it out in the ID, and then approach the managers to buy a license. This is one of the ways, which works quite successfully. Of course, we have... also people who approach us directly from the higher manager levels and say, yeah, we're interested in the solution, we want to do a proof of concept, and so on. And then we start with a team, and then they expand to more teams within these accounts. This is our usual approach. But we are not really restricting it to to particular verticals it's more that there are certain problems that people want to solve like they want to re-architect code bases and to realize they don't have enough tests and then they're looking for these kind of solutions and yeah these are the kind of customers we're looking for

Yeah. Thank you so much for your time. I really appreciate it. It's great to learn. I think you've got sort of an application and a you know, a strategy, the strategy of sort of making it hands-off, right? I think that's a unique thing that we've seen about DiffBlue. Out of all the AI companies that are working right now, most are sort of still at the stage where they require human input and checking, right? And I think the application of just being able to have an AI system that does things for you automatically is a pure benefit, right? It's something that's very different from many AI systems that are out there. So I think it's super interesting. And so any, you know, any upcoming announcements or, you know, what are you looking for in 2024? You know, anything that people should know about DiffBlue? What are you guys sort of working on that's next level? So, we're exploring integration with more systems, such as CI providers like GitHub Actions, GitLab, or Jenkins, which are quite popular. Our aim is to simplify the deployment of our solution within these systems. Additionally, we're looking into expanding compatibility with more IDEs and similar tools. Naturally, we're continuously striving to offer our users more comprehensive and effective tests.