At the beginning of April, DALLE-3 received a new update, which is inpainting – a feature that allows for editing photos, including fragmentarily.

Inpainting & DALLE

What exactly is inpainting? In the context of generative models, it is a method that allows for photo editing using prompts – adding elements and editing selected parts of the image. This option is useful from a business perspective – thanks to inpainting, anyone can become a graphic designer and easily modify existing assets for current needs.

What is DALLE-3? It’s a generative model from OpenAI that allows for generating images. It’s available through ChatGPT on the GPT-4 subscription.

So are we talking about another breakthrough under the OpenAI banner? Not this time. Inpainting isn’t something new – we’ve been using it for a while in the Fooocus tool (watch our tutorial and try out this tool yourself!). And perhaps DALLE-3 does it better?

To find out, we decided to compare these two tools and share the results with you.

Test

Before presenting the results of our experiment, it’s worth remembering the following assumptions – both models were likely trained on different data, so they may behave differently for different domains. This may affect the outcome of our tests.

How did we conduct the experiment? We generated two images using DALLE-3, and then edited them using identical prompts for the same selected areas.

Initially, we asked DALLE to generate a picture of a Martian desert and a monkey holding a water bottle:

Inpainting on Mars

First, we tackled the desert visualization. We asked both models to generate a modern city with skyscrapers in the highlighted center. And here’s a false start – at the first attempt, DALLE-3 returned “nothing” – there were no changes in the graphic.

On the second attempt, DALLE managed to “add” skyscrapers. Below is a comparison of the effects we obtained for the same prompt from DALLE and Fooocus: 

Fooocus seems to handle much better when precisely marking the area where we want to add a new element. DALLE lost quality in its inpainting in this case.

It is worth adding here that DALLE handles the addition of specific elements well (from the prompt) but in any place on the original image – as shown in the example below, where we decided not to mark the area for editing (which is actually the main advantage of inpainting, but we wanted to give DALLE a chance).

In the second iteration – we asked for a spaceship. DALLE started with its old trick – no spaceship appeared, although DALLE insisted that, “The Mars landscape now includes a sleek, futuristic rocket ship poised for launch, adding an element of human exploration and progress to the scene.” To be sure, we asked him the same thing again, but he was obviously tired and didn’t want to cooperate anymore. 😉

On the other hand, Fooocus handled the task very well and added what was needed to the picture.

Monkey, Exploring AI’s Relationship with Animals

Then we decided to play with our little, friendly monkey. Cowboy hat or military helmet? No problem for both models. Although in this case both models returned correct figures, those from DALLE seem to better fit our model.

The second approach – we wished to add a second monkey trying to steal the bottle. The models easily generated the animal, but definitely understood differently which bottles we meant.

So, DALLE or Fooocus? Our conclusions

What conclusions can we draw from our test? Inpainting in DALLE 3 doesn’t seem to have the same “wow” factor we’re accustomed to from OpenAI achievements. For us, it doesn’t surpass Fooocus in any way, especially considering that it’s also available as a free tool.

During our tests, DALLE failed twice and simply didn’t generate anything. The first “Martian” city it created in the highlighted area didn’t impress and just didn’t fit the scene. In this regard, Fooocus clearly outperformed DALLE.

Analyzing the monkey example – both models performed well, and DALLE surprised with almost perfect matching of additional elements. Where do these differences come from? The answer may lie in the distribution of data on which the models were trained. Simply put, the monkey example could have been better tailored to what DALLE was trained on earlier. Fooocus seems to better handle realistic additions and implementations.

Equally important as effectiveness – both models provide great fun to use and under the right conditions can effectively support us in graphic work. However, for us, Fooocus remains the favorite.

Boost your knowledge and skills
with our AI Newsletter