If possible, could the official team consider making it compatible with ComfyUI?

[raysers]

The FLUX model is quite unfriendly to Mac users, but fortunately, there's MFLUX, which is the only successful attempt I've had. If this project could be integrated into ComfyUI, it would be much more convenient than using the terminal. It's possible that diffusionkit is also working on something similar. I believe this could be the springtime for Mac users.

[azrahello]

I’m starting to love this project (and also DiffusionKit), considering how many things are coming out for mlx. I’m beginning to think about something similar to ComfyUI, which allows heterogeneous and complex workflows, helping you understand what’s happening without diving too much into programming, but entirely designed for mflux. The way mflux and DiffusionKit work is fantastic. ComfyUI might become a bottleneck since it’s primarily developed for CUDA. If I had the skills, I would have already started working on it. :(

[raysers]

I’m sorry I can’t be of much help since I’m just a coding novice myself. Ever since I started using ComfyUI, I haven’t opened WebUI again. I believe ComfyUI has great potential due to its high customizability. For example, with the MFlux project, if it could run as a new plugin within ComfyUI, I could use it in ways like this: leveraging the existing plugins in ComfyUI, it might be possible to integrate MFlux into workflows for reverse-engineering prompts, or into workflows for background removal... There are many other similar possibilities worth exploring for users, which is why I’m amazed by ComfyUI’s potential. However, I now see that participants in the MFlux project are researching WebUI-based content. If that can be realized, that’s also good, although I believe its extensibility is far less than ComfyUI. But we are just users, not code contributors, and we always tend to take things for granted, not realizing how fortunate we are to have come across this project. Thanks to MFlUX

[filipstrand]

@raysers @azrahello These are really kind words, thank you very much :)

Speaking for myself, I have not personally used ComfyUI yet but it does look very powerful and there seems to be a big community around it (sharing workflows etc). Currently, I have no idea how much of an effort it would be to integrate MFLUX as a backend (if it is even possible etc.) and what kind of features would be needed to be supported by us to be properly integrated, but it is definitely an interesting avenue to explore. If similar kinds of integrations to ComfyUI have been made before, it would be great to know and read up on them.

At the moment, there are still things prioritised on the backend/core part of the system (most notably LoRA fine-tuning support and some other features) which will be the focus near term from my part as well as overseeing incoming additions and contributions. However, once the backend feels more fully fleshed out and stable, then a natural next step would be to consider integrations with things like ComfyUI or similar. (Of course, if there is a good PR proposal with a Comfy UI integration, it will be considered).

It is however very valuable to get your opinions here about supporting a more "basic" webUI vs ComfyUI to get some sense of what to prioritise longer term since there are many different possible routes to take. I think that @anthonywu has made some good points about the role of mflux (the core) vs different possible frontends. Even though mflux might not host a whole family of different frontends itself, it is probably a good idea to also prioritise features that will enable future GUI support, such as stepwise image generation #59 etc.

[raysers]

Thanks to 大佬 @filipstrand for clearing up the confusion. I see that you all are still working hard to continuously improve Mflux, and I sincerely admire and extend my deep gratitude. Of course, you should first focus on stabilizing the backend according to your current work plan, but 大佬’s comments on the frontend UI also make me excited. Thanks again!

PS: In Chinese, “大佬” refers to a master or expert in a certain field. I could only think of this word to express my respect—apologies for any confusion.

[n0kovo]

PS: In Chinese, “大佬” refers to a master or expert in a certain field. I could only think of this word to express my respect—apologies for any confusion.

Wholesome GitHub

[raysers]

Hello everyone, I have started working on the porting process, aiming to implement a simplified version of Mflux in ComfyUI. Currently, I have completed the image generation workflow using a 4-bit quantized model in ComfyUI. The next step is to debug the integration of LoRA and ControlNet, which might be beyond my expertise, but I will do my best. If Mflux is ported as a ComfyUI plugin, it will need to follow the plugin installation rules of ComfyUI, which may require releasing a new project named "Mflux ComfyUI" — this is coming soon.

[raysers]

Hello everyone, I have started working on the porting process, aiming to implement a simplified version of Mflux in ComfyUI. Currently, I have completed the image generation workflow using a 4-bit quantized model in ComfyUI. The next step is to debug the integration of LoRA and ControlNet, which might be beyond my expertise, but I will do my best. If Mflux is ported as a ComfyUI plugin, it will need to follow the plugin installation rules of ComfyUI, which may require releasing a new project named "Mflux ComfyUI" — this is coming soon.

Hi, this has started, and everyone can try it out via

https://github.com/raysers/Mflux-ComfyUI

However, it currently only has the basic text2img functionality. Although there are LoRA parameters in the nodes, they are not yet usable. LoRA is an urgent issue that I can't solve on my own. I hope someone experienced can offer guidance or even collaborate on the development.

[raysers]

Has anyone tried it? I might have overestimated the demand for using mflux on ComfyUI. So far, only 20 people have tried to clone it. But I’m still updating it. I’ve now added ControlNet and updated some workflows. For those who want to try it, you can git clone, and for those who have already installed it, you can git pull. ControlNet is something mflux already had, I’ve just implemented its original features on ComfyUI. As for the early exploration of those workflows, that's why I’m persistently trying to port mflux to ComfyUI—I’ve always believed that mflux can only fully realize its potential through ComfyUI. Greetings to everyone again!

[azrahello]

@raysers I’ve tried it, and I found it very intelligent as a development. The only thing I would make a PR for is to have the models quantized to 8-bit (this first and foremost, and perhaps, as an option, the ability to choose whether to use mflux at 16 or 32-bit). Other than that, it’s magnificent—let’s hope it grows and expands (though I understand that might not depend on you). Inpaint support is missing, which is also missing in mflux, although mflux has DiffusionKit, which doesn’t support LoRa, while mflux does. In my opinion, it’s a bit of a mess :P

“PS: Now that previews are experimental in mflux, they could be implemented in the ‘generator’. Also, it would be more functional (since it introduces fewer chances of errors) if, when I select ‘dev’ in the generator, only the dev paths are shown in the list. I imagine this could be done with an ‘if’ loop that checks for the word ‘dev’ in the file name.”

[raysers]

Thank you @azrahello for letting me know that I'm not working alone. Compared to the ComfyUI version of DiffusionKit, my preview issue is caused by a difference in the way it's called. His code goes deeper into the core logic, using latent decoding directly. On the other hand, I’m calling the image generated by MFLUX's FLUX1 and converting it back into a tensor to achieve the preview. As a beginner, I can only work with these ready-made image generation integration files in MFLUX. I’d say I’m not professional enough yet.

Honestly, I really hope that once the official team has some free time after completing their current workload, they could consider working on this porting project. When that happens, my temporary version can step aside. As @filipstrand 大佬 mentioned, "it would be great to know and read up on them." If a reference example is needed, I would still recommend DiffusionKit's ComfyUI version, which can be found at https://github.com/thoddnn/ComfyUI-MLX. This port is highly successful, with clear steps from model loading to latent input and VAE decoding, creating foundational nodes that are easy to follow. It’s like the difference between a notebook and a custom-built PC—where with the latter, we can freely choose the parts we like. This might also align with the core philosophy of COMFYUI.

[raysers]

Hello everyone, Lora is coming now. Hey, @azrahello, you were right. In your last reply, you mentioned giving users more options, allowing them to freely choose between the full version (dev and schnell), or 4-bit or 8-bit quantization. So, I restructured the model loading mechanism to align with the official implementation. Once I finished that, the Lora issue was resolved as well. You can refer to the discussion in #47 for more details.

I think I might have taken a detour earlier. Due to the low performance of my machine, I was stuck in a mindset of only using 4-bit quantized models to balance quality and performance. But now I've realized that the official design offers more flexibility. Users with high-performance machines might prefer the full version or at least 8-bit quantization.

Anyway, Lora is here now. This is a significant feature implemented in ComfyUI using mflux. I believe we're not far from achieving the full version’s functionality. The only thing missing is the model-saving part. Additionally, the official team is still rolling out new features, and I’ll try to keep up within ComfyUI. Cheers!