ZBDongle-E flow control

[darkxst]

Nice work getting these builds up and running.

I dont think the ZBDongle-E supports hardware flow control and needs to use software flow control. In fact i recall reading somewhere the CTS and RTS lines are not connected on this device.

I havent tested your specific builds yet, however I have been running Skgsergio's RCP builds for ZB-GW04 v1.1 on my ZBDongle-E which works well. His ZB-GW04 v1.2 builds with hardware RTS/CTS do not work on the ZBDongle-E in my testing.

CTUNE should probably also be set to 128 on this device, per this comment from iTead
grobasoz/zigbee-firmware#28 (comment)

[ksjh]

Thank you for the hints. This is very helpful. I will use a different CTUNE value in the next build. But why not simply use 133, when this was the best value in the tests? An individual value per model is not problem at all with this build infrastructure.

Regarding the HW flow control on the ZBDongle-E: I have several of those, and mine work very well with my firmware. Can you please explain what kind of problems you were experiencing with Skgsergio's builds? I tried his v1.2 with 115200 baud on my ZBDongle-E before I built my own version, and, as far as I could see, it worked for the short time I tested it.

Are we talking about the current version of the dongle, Zigbee 3.0 USB Dongle Plus, Model ZBDongle-E, EAN 6920075777659?

[ksjh]

So, I just built a version for the ZBDongle-E without hardware flow control for those who need it (ending in -none). Also, all ZBDongle-E versions now use a HFXO CTUNE value of 133.

[ksjh]

I read the mentioned answer by itead again. I am really not sure if 128 or 133 is the correct value, but since they stated to use 128 for best performance, I initialized a new build, now with the value 128.

[darkxst]

Are we talking about the current version of the dongle, Zigbee 3.0 USB Dongle Plus, Model ZBDongle-E, EAN 6920075777659?

Yes I am using the current -E dongle.

I couldnt get the Silicon Labs addon to connect to the dongle with v1.2 build, but then it worked straight away when I switched to v1.1. I will test again though and see.

[ksjh]

So did you flash the RCPMultiPAN or the EmberZNet version? I am currently using the RCPMultiPAN version with 230400 baud and hardware RTS-CTS handshaking in an experimental setup of Home Assistant on x86-64 with the Silicon Labs Multiprotocol add-on v1.0.2 and zigbee2mqtt (edge). Therefore, I set the SONOFF Zigbee 3.0 USB Dongle Plus V2 integration to be ignored. Perhaps there is just something wrong with the EmberZNet firmware? I will check this version again.

[darkxst]

I am using the RCPMultiPAN firmwares and Silabs Multi addon, with test instance of HA running in a VM.

Just tested again with 4.2.2 builds
RCP 115200 hardware: just loops with "failed to connect to secondary"
RCP 230400 hardware: this connected successfully and seems to be working fine

Ive not tested the EmberZNet NCP firmware.

[darkxst]

Also what do you use to flash these? Universal Silabs flasher is failing to probe via CPC once I have one of your firmwares flashed, where as I pretty sure Skgsergio's builds were working ok with that. I ended up having to use the physical boot button and Elelabs flasher to upgrade from these.

[ksjh]

I use the same NabuCasa Universal Silicon Labs Flasher when trying my firmware builds. No problems with this whatsoever. The only three things to make sure:

• the --baudrate needs to be adapted to the baud rate of the firmware that is currently installed on the ZBDongle-E
• the options --allow-cross-flashing and --allow-reflash-same-version might be needed, depending on the currently installed firmware
• when the new experimental OpenThread RCP builds (Thread only, no Zigbee support) are already installed on the dongle, the current version of the NabuCasa flasher will not work. So, do not flash these versions, unless you know what you are doing and know how to flash the dongle by other means.

[P1X3L8]

Is there any difference in using the NabuCasa flasher vs a terminal and xmodem?

[ksjh]

Is there any difference in using the NabuCasa flasher vs a terminal and xmodem?

Well, the results should be the same. The NabuCasa flasher can start the bootmanager of the dongle without pressing a button. It also has some additional checks and functionality (e.g., changing the EUI-64 = IEEE address). After starting the bootmanager, the NabuCasa flasher also uses XMODEM to upload the firmware, just like manually with a terminal.

[darkxst]

• when the new experimental OpenThread RCP builds (Thread only, no Zigbee support) are already installed on the dongle, the current version of the NabuCasa flasher will not work.

This should be fixed in v0.0.10 of NabuCasa flasher, however it does rely on patches to the Openthread SDK so will only work with firmware built using those, so probably wont work with other random community built firmware.

[ksjh]

Thank you for the hint. I have seen that there have been changes to the NabuCasa flasher regarding this point. Since I ported their SDK patches, it should also work with this firmware builds, but I have not tried the new flasher yet, so I cannot really be sure. I will try it and report back.

[FredWst]

Hi,

I just discover your build. 👍 nice job.

I'm using Zigbe2mqtt edge, rcp-uart-802154-zbdonglee-230 -> not working fine. (just like SkyConnect)
Looking at log and seems trouble come from Z2M. Both dongle not working.
Zigbe2mqtt with ncp-uart-hw-zbdonglee-230 works fine.

Hope 460800 will come soon.

On Debian, I use last build universal-silabs-flasher from NabuCasa.
In terminal sudo universal-silabs-flasher --device /dev/ttyACM0 flash --baudrate xxxxxx --firmware fffffff.gbl --allow-cross-flashing

Fred

[ksjh]

I just added builds with 460800 baud, but did not test them myself, so far. Feel free to do so...

[ksjh]

Hi Fred, if you want to use the rcp-uart-802154-zbdonglee-230, you need to use a software component to handle the Zigbee stuff called zigbeed. The rcp-builds are not intended to be used directly with zigbee2mqtt. They can easily be used with the Silicon Labs Multiprotocol add-on for Home Assistant. Since zigbee2mqtt currently does not support Thread/Matter, it does not make much sense to use the rcp builds in a pure zigbee2mqtt environment without the add-on. But when you use Home Assistant and the Multiprotocol add-on with the RCPMultiPAN builds, you need to configure your Home Assistant port differently. Something like

port: tcp://core-silabs-multiprotocol:9999
adapter: ezsp

should do the job.

[FredWst]

@ksjh

My config was like you said. But not working.
My devices are showed and not working.
I will try again.
Maybe I should pair again my devices ?

homeassistant: true mqtt: server: mqtt://core-mosquitto:1883 user: mqtt_adm password: ********* keepalive: 60 serial: port: tcp://core-silabs-multiprotocol:9999 adapter: ezsp frontend: port: 8099 advanced: log_syslog: app_name: Zigbee2MQTT eol: /n host: localhost localhost: localhost path: /dev/log pid: process.pid port: 514 protocol: udp4 type: '5424' channel: 25 log_level: error devices: '0xa4c13823f6660add':

Fred

[ksjh]

I am really not sure what is happening here. I have the same configuration running on an experimental system.
The zigbee2mqtt docu mentions that you might need to reboot the devices when changing the coordinator hardware. Perhaps you can try this. If it does not work, you can try to re-pair at least one device first and see if this single device works.

[FredWst]

Look at pictures.
Many LQI error bad quality, just like if radio 20db was not active.
My PC is in my garage. (Working well with firmware NCP)

[ksjh]

Thank you for the feedback. I will check what I can do.

So you use the same ZBDongle-E in in the same location in both cases, once with my NCP firmware, and once with my RCP firmware. With the NCP firmware, you do not get LQI errors. But when you use the same ZBDongle-E with my RCPMultiPAN firmware, you get lots of LQI errors. Is this the case? Did I understand you right?

[ksjh]

I just compared the configuration of the PA in the source code of the EmberZNet and the RCPMultiPAN firmware. They are completely identical. It could be an error in the build process, but i have found none, so far. I will try to search a little bit more. But I also have had no detailed look into the Zigbee daemon zigbeed. Since it is a software layer between zigbee2mqtt and the hardware and only active when using the RCP builds, it could also cause some effects.

[FredWst]

After pairing all my devices, all working fine.

So RCP and NCP firmware work fine.

Fred

[ksjh]

Glad to hear that. I intend to close this issue soon, unless there are more related problems. If there are other unrelated problems, please open a new issue. You can also use the discussions on this page if your topic is not a firmware issue.

Thank you.

[FredWst]

Hi,

Why LQI NCP are different from LQI RCP ?

RCP is always 255, with NCP according to room, it is between 255 and 100.

Fred

[ksjh]

Similar to the issue about the LQI errors, the root cause of this problem could be multiple things:

• the firmware itself
• the Zigbee daemon zigbeed
• zigbee-herdsman, see issue #677 for a similar problem with other hardware.

The firmware source code is essentially published by Silicon Labs, the modifications are minimal, just to support the respective hardware. If there really is an error in the reporting of the LQI in the firmware, I suspect that it is already in the original source code. But I guess it is more the way zigbee-herdsman communicates with the hardware in this special case.

Please also note that the RCPMultiPAN concept is still in an early stage, I would not really consider it as production ready. In a larger setup, it might make much more sense to use two communication units (dongles), one exclusively for Zigbee (with EmberZNet), and one just for Thread (with OpenThreadRCP, also in an early stage), even if the lower layers 1 and 2 (essentially PHY and MAC) are IEEE 802.15.4 in both cases. But since the higher layers differ significantly, it might make sense to handle both protocols completely separately.

[FredWst]

Thank you for answer.

"I suspect that it is already in the original source code" I made the test with SkyConnect and was having same issue with ZBDongle-E.

Fred

[ksjh]

That is very interesting. Thank you.

Have you also tried the SkyConnect firmware provided by NabuCasa? It should be the same, since I forked their project, but did not change anything for the SkyConnect.

[FredWst]

I tested with :
SkyConnect firmware NabuCasa.
ZBDongle-E your last build.

Fred

[darkxst]

Back to the original topic of this discussion, I inspected the ZBDongle-E PCB.

On the serial chips (Sonoff seem to be covering chip shortages by allowing one of two different serial chips to be populated on the PCB)
CTS is not connected to anything
RTS appears to be connected, along with DTR. This looks very much similar to an auto-downloader circuit that is common on ESP devkit boards for triggering the bootloader.

On the Silicon Labs EFR32MG21 chip:
PD03 and PD04 are not connected

Thus it would seem that these devices are unable to physically support any hardware flow control. The fact these builds are working, probably just means that its falling back to either no flow control or software flow control.

It would be simpler and probably save some confusion to just have one set of builds for ZBDongle-E only using software flow control.

[ksjh]

Thank you, you are completely right, I looked at my ZBDongle-E and found the same configuration. It is marked as V1.0 2022-03-22 and uses a WCH CH9102F. I found the info about the hardware flow control pins somewhere on github, but did not visually confirm it with my hardware, I just tried if it worked, and it did. Which, by itself, is strange. Of course, I will have to change the builds.

[darkxst]

This looks very much similar to an auto-downloader circuit that is common on ESP devkit boards for triggering the bootloader.

This is indeed the case, RTS/DTR signals can be used to reset the dongle into Gecko Bootloader. Which should work even if a bad firmware is flashed to the dongle.

[darkxst]

Yes that is the same hardware revision as the one I opened with CH9102F.
The unpopulated chip appears to be for the CH92102X.

[ksjh]

As explained by MattWestb here, you would only notice that the flow control is not working when the buffer was full. This is due to the fact that both RTS and CTS are normally pulled to ground, as it seems, and they are active low signals, so normally always asserted and never cleared, even if they should be (e.g., due to a full buffer). So, the firmware with hardware flow control always allows to send and works, just like a firmware compiled without flow control.

But I am somewhat uncertain which data rate (baudrate) to select. Should we continue to provide all between 115200 and 460800 even without flow control? I think, as long as the hardware is in principle well designed, higher data rates should be no problem. The probability of a buffer overflow should in most cases be lower with a higher data rate. One issue I could see would be OTA firmware updates, where the host might then send data to the wireless side much faster than IEEE 802.15.4 could handle. But in normal cases, I expect more traffic from the wireless side to the host, and thus a higher data rate to the host would help to keep the buffer empty.

[darkxst]

So, the firmware with hardware flow control always allows to send and works, just like a firmware compiled without flow control.

There is an issue though where the dongle doesnt come back up after a reboot of the host system when using hardware flow control. Atleast in my testing this didnt happen with the -none build without flow control.

Should we continue to provide all between 115200 and 460800 even without flow control?

All testing so far has been without flow control, and things have been working. I would keep building the same speeds. Not rally sure what would happen when the host speed exceeds the 802.15.4 datarates for OTA update, but its going to be pretty obvious if there is a problem. Worth testing though.

Also the NCP Firmware has the option for software (xon/xoff) flow control, which is I presume what iTead are using in there builds. RCP doesnt seem to have this option.

[ksjh]

Could the startup issue somehow be related to the RTS/DTR bootloader reset handling?

[darkxst]

I wouldn't think so, given its only happening with hardware flow control builds. I am thinking its more related to the disconnected CTS/RTS lines, either the connection is stuck open or something is waiting for a signal from those.

However I will note my 'host' is running in a VM so a reboot, is not necessarily going to reset the dongle, like reboot of a physical host would.

[darkxst]

Should we continue to provide all between 115200 and 460800

I'm wondering if should only build baudrates that will be autodetected by Universal-Silabs-Flasher per the list below:

DEFAULT_BAUDRATES = {
    ApplicationType.GECKO_BOOTLOADER: [115200],
    ApplicationType.CPC: [460800, 115200, 230400],
    ApplicationType.EZSP: [115200],
    ApplicationType.SPINEL: [460800],

Unless there turns out to be issues with the 460800 rates without flow control, but I not noticed any yet.

[darkxst]

I hooked up the RTS/DTR bootloader reset and opened a PR upstream
NabuCasa/universal-silabs-flasher#31

[darkxst]

Should we continue to provide all between 115200 and 460800 even without flow control?

Here are some benchmarks of RCP Multipan with and without hardware flow control. From those results I would suggest we actually want higher speeds with no flow control, speeds < 250kbps are probably going to result in more packet loss (they didnt even report results for no flow control at 115k or 230k!).
https://www.silabs.com/documents/public/application-notes/an1385-multi-pan-rcp-performance-test.pdf