We're live on Product Hunt right now! Come support our launch

rocket logo for website pages

Winner of the 'Embedded Analytics Solution of the Year' at the '24 Data Breakthrough Awards.

AI Developers - How do they Perform?

Contents

Looking for an embedded analytics tool?

Embeddable embeds in minutes, loads in milliseconds and extends infinitely - enabling you to quickly deliver lightning-fast dashboards that feel native in your app.

Find out more

Over here at Embeddable, we recently spent some time playing around with a new tool you may’ve heard of before, Lovable, an AI code generation platform that can scaffold out an app in a matter of minutes from a natural language prompt. I thought it was pretty interesting, so I dug into it a bit more in my personal time over the last few weeks, using it to start a small project.

I haven't paid anything for it, yet. I’ve just been using my five free prompts a day. Since so many of the folks who use Embeddable or are interested in doing so are also curious about AI tools, I wanted to write up some thoughts about my experience with it.

Let’s channel 60’s-era Clint Eastwood for this one …

The good

Like with a lot of AI stuff, the first few times you work with it, Lovable feels like magic! It handles a ton of out-of-the-box stuff for you and saves a lot of setup time. It does a great job of converting natural language prompts into functioning code, and the initial code itself is pretty clean and well-structured.

When you ask it to fix bugs, it almost always does so. It regularly introduces stuff that you hadn't asked for yet, but likely were going to (eg error/success toasts). This is a double-edged sword sometimes, but is often useful. It's remarkably fast, and it even comments the code fairly well.

Lovable absolutely saved me time over doing the early React scaffolding myself, and I liked that it erred toward Vite rather than Next.js, as it’s a more lightweight solution (you could explicitly tell it to begin scaffolding with Next, if that was your preference).

It also did a good job of identifying the latest versions of packages and using those, so I didn’t find that I was stuck with older versions of things like my preferred design library.

The Bad

Like with a lot of AI stuff, the longer you work with Lovable, the more you realize its limitations.

It’s very good if you want to work with Tailwind and a handful of UI libraries that use Tailwind. It struggles if you tell it you don't want to use Tailwind, and keeps repeatedly inserting Tailwind and things that depend on it, even after you’ve told it not to. It also had trouble understanding how to style my preferred design library, and added a significant amount of CSS that didn’t do anything.

Lovable often introduces basic bugs (like writing code that uses an import but never including the import) and then charges you a credit to fix them. It is not terribly good at scaffolding for back-end solutions that aren't the one it supports out of the box.

If you want to avoid these kinds of issues, you need to give increasingly longer prompts expressly telling it what to keep and what to get rid of.

The ugly

My experience so far is that the longer you use Lovable on a single project, the worse it performs.

Because AI isn't good at retaining long-term context, it will regularly get rid of stuff you asked for previously while adding in new stuff you asked for more recently, requiring further prompts to fix things or (in one case, for me) outright reversion. Pretty often, it will get rid of stuff you want while adding stuff you never asked for and don't want.

I also found that Lovable leaves a lot of artifacts in the code, for example filling package.json with imports that don't get used, or in my case at one point building out an entire folder full of UI components (buttons, inputs, etc) that were not used anywhere.

It reached a point with my project where I could not reliably get it to move further without introducing bugs and code cruft with every prompt. So I stopped using Lovable and just switched to hand-editing.

I've ended up spending a lot of time removing artifacts, cleaning up code, and rebuilding stuff to work with the back-end I want instead of the one Lovable seems to prefer.

Takeaways

I want to be very clear to our readers: I'm not anti-LLM! I use Github Copilot daily and as an "autocomplete but better" solution, it's very useful. It needs supervision, but it's still pretty great!

Here at Embeddable, we also employ AI for natural language search through our docs, FAQs and roadmap, and it’s very useful for our clients. We’re actively investigating how else we can incorporate AI into our products while ensuring that their overall integrity, flexibility, and power isn’t compromised.

That said, I would be very hesitant about endorsing any policies like "build with Lovable first, then tweak" based on my own experience with the platform.

It's a fantastic "vibe coding" tool for cranking out fun little projects, but without significant oversight, my experience is that it starts to get away from you pretty quickly. You end up spending a lot of time figuring out what the code is doing and then correcting it.

Overall, I remain curious about AI-assisted coding and think there are likely innovations coming that will address a lot of the issues that I’ve brought up in this post. I think we’re still a ways out from AI being a trustworthy way to generate a tight, secure codebase on a larger scale, but if you’ve been thinking about using Lovable to build a small project, or put together the early structure for a bigger one, it’s definitely worth a look!

Looking for an embedded analytics tool?

Embeddable embeds in minutes, loads in milliseconds and extends infinitely.

Find out more