Updating 4343W wifi firmware for our module via OTA

Tip / Sign in to post questions, reply, level up, and achieve exciting badges. Know more

cross mob
rashc_2727106
Level 4
Level 4
10 likes received First like received First like given

We have an existing product already on the market that uses a module with the 4343W silicon and shipped with bootloader and the original OTA (not OTA2) code from WICED SDK 4.1.1.  We would like to OTA to an updated version based on 4.1.3, which has an updated wifi firmware image with the KRACK fixes.  The APIs, DCTs, and bootloader are the same, so it should be simple.

The normal WICED OTA solution works fine for us for updating our application code.  But it does not update the wifi firmware for the 4343W chip.  

I have spent time searching the wiced and wwd code base, and the forums here without finding any obvious reference to a "standard" WICED solution for this problem.

I can, of course, roll my own solution in our app by downloading the new 4.1.3 wifi firmware bin into a new location in external sflash, and then using the existing wwd APIs to

reset and reload the module. 

But I cannot be the first developer to have this need, so I'm assuming that I'm missing some document, demo, snip, or forum discussion where this is already addressed.

Can someone please point me in the right direction?

Thanks,

15 Replies
Zhengbao_Zhang
Moderator
Moderator
Moderator
250 sign-ins First comment on KBA 10 questions asked

Hello Rsh:

  I  just used 43362WCD4 to do a test, below are the steps:

1.  use OTA app to do a upgrade , the upgrade elf file is test.mfg-test-BCM43362WCD4.stripped.elf

     and do a wl ver test by command line ,  make sure it is a MFG test  version FW.

2. then i copied the normal FW to replace the mfg fw in the resources/firmware , re-compile the test.mfg-test app.

3. use OTA to do an update again,  then I find the FW goes to the normal one from the command line.

so from my thinking you need to copy the 4.1.3 wifi fw into the 4.1.1 folder,  then re-compile.

do the OTA upgrade, the FW should be upgraded.   By the way my studio environment is wiced studio -6.0  .

0 Likes

zhez wrote:

Hello Rsh:

  I  just used 43362WCD4 to do a test, below are the steps:

1.  use OTA app to do a upgrade , the upgrade elf file is test.mfg-test-BCM43362WCD4.stripped.elf

     and do a wl ver test by command line ,  make sure it is a MFG test  version FW.

2. then i copied the normal FW to replace the mfg fw in the resources/firmware , re-compile the test.mfg-test app.

3. use OTA to do an update again,  then I find the FW goes to the normal one from the command line.

so from my thinking you need to copy the 4.1.3 wifi fw into the 4.1.1 folder,  then re-compile.

do the OTA upgrade, the FW should be upgraded.   By the way my studio environment is wiced studio -6.0  .

The wlan firmware can be located at 3 possible places:

1. built-with your application firmware

   (in this case, the application firmware will be very big)

2. in resource filesystem

3. in a separate section (by setting MULTI_APP_WIFI_FIRMWARE)

Your test only works for case1.

Thanks zhez​ for trying the experiment.  Appreciate the support. It's good to know that the WWD_DIRECT_RESOURCES solution that you tested does work.

But alas, we're using MULTI_APP_WIFI_FIRMWARE so our WiFI firmware is located in a separate section in external sflash, hence that solution doesn't work for us as axel.lin_1746341​ helpfully pointed out.

0 Likes
AxLi_1746341
Level 7
Level 7
10 comments on KBA 5 comments on KBA First comment on KBA

One important thing you need to think about: What happened if ota wlan firmware fails?

If your device has enough space to store 2 copy of wlan firmware, it should be fine.

If your device has limited space and only one copy of wlan firmware, the device lost

wifi functionality if ota fails.

Actually, that is the point I pointed out why the clm_blob file causes problem.

With clm_blob file in resource filesystem, either resource filesystem or wlan firmware update failure

the device will be dead.

0 Likes

Thanks axel.lin_1746341​.  You raise good points about both the wifi firmware update, and the corresponding possible failure path (eventually in the 5.2 sdk) when we have to update clm_blob as well.

At the moment with our 4.1.x solution, we do have room for two complete wifi firmware images in our external sflash.   I can see where the normal wifi image gets loaded to the module during boot/init inside the function

WICED/WWD/internal/bus_protocols/wwd_bus_common.c::download_resources(). 

So if I have to roll my own (still hoping I don't), then I can use the relevant API calls (possibly with a very minor hack).   I was going to do something like the following from within our app:

  1. Download new wifi firmware image and place it in a new section in sflash (not overwriting the existing image location).
  2. Save some persistent indicator that we're trying to update wifi firmware
  3. Reinit wiced / wwd passing the new location (based on that persistent indictor) to a (possibly hacked) version of the download_resources() code.
  4. Check at several stages for succesful module bringup and WiFi connectivity.  If any fail, then revert to the original wifi firmware image and reinit (and send the server an error).

The implementation does not look that hard, but the fail-safe testing will be time-consuming.

I'm still hoping that Cypress or someone can point me to an existing solution for this issue. 

axel.lin_1746341​ Would I be correct in inferring from your reply that no example or demo of such a solution exists?

0 Likes
Anonymous
Not applicable

According the CYW43438 data sheet,  BCM43438 only has 640KB of ROM,  How can you program the ROM?

pastedImage_0.png

0 Likes

terrance.peiris_2100836​  I am still new to WICED so take this response with due skepticism, but after perusing the 4343W data sheet (4343w-DS_002-14797_0I_V) it looks to me like the 640kB of internal ROM is really ROM and not programmable.   Section 12 suggests (but is a little ambiguous) that the ROM contains the DHD host driver, which is responsible for interacting with the host to load the wifi module firmware into the 512kB RAM.

0 Likes

For anyone facing the same issue, I did end up rolling my own solution.  We re-used the multi-app support pieces in WICED to save wifi fw images to App1 and App2 locations (App0 is used for app OTA updates).   It turned out to be more of a pain than expected because some of the functions deep in the WICED guts have some undocumented restrictions that we had to work around.

If there's interest, I can post some details.

Thanks for the offer rsh_2727106

Adding a few members of our Apps Team: grsrriyawwferroy

0 Likes

Looks like there is some interest, so here is a description of what we did.  Note that there are two solutions sketched out below - a basic one that will work if you have room in your external flash for two complete wifi *.bin images, and a more complicated one that stores compressed *.bin images in external flash.  We started with the first but ended up needing the second. If you only need the basic solution, you should find it pretty easy. But it was in the compressed-image solution where we ran into two unexpected issues inside the WICED code.

This ended up being a long post, so a disclaimer before you read too far.  Our solution does not deal with the sdk v5.2 resource changes (e.g. clm_blob management).  Our focus was getting our wifi firmware from 4.1.1 to 4.1.3 in the field. 

Background

I'll start with a little more background on the problem.  

Our hardware uses an LSR module with the 4343W chip interfaced via SDIO to an STM32F4xx microprocessor.  We already had many devices (~1000's) in the field with working app OTA.

Our app was initially released with SDK v4.1.1 and uses MULTI_APP_WIFI_FIRMWARE with the default external flash map:

AppLUT

Factory Reset wifi firmware image - stored as a *.bin image.

Factory Reset app  firmware image - stored as a stripped elf file.

App0 - OTA app update image

   

Since our wifi firmware OTA is based on that existing app OTA code, we'll start by sketching what app OTA looks like for us. 

The app would communicate with our server and identify the URL for a new app firmware image, connect to that URL via HTTP, download the image in chunks, and copy each chunk into external flash at the correct App0 location using the WAF framework API.

Specifically, for each chunk we would get a fixed-size 4096 byte block of the app image from the HTTP download and copy it to external flash using

wiced_framework_app_open(DCT_APP0_INDEX,  &app)

wiced_framework_app_write_chunk(&app, ...)

and then once we were done with downloading all the chunks, we call

wiced_framework_app_close (&app);

wiced_framework_set_boot( DCT_APP0_INDEX, WICED_FRAMEWORK_LOAD_ONCE )

to finish the copy and force the next reboot to load the new app code.

The app also receives the new app image download checksum from the server and verifies it during the download before we commit to rebooting into the new image.  You’ll want to make that final decision to commit atomic so that if anything interrupts the process before that verification you can still recover to the old application.

Basic wifi fw ota

Our requirements for the new wifi fw ota:

  1. We had only about ~6k bytes left for new code, so we had to reuse as much existing code as possible.
  2. The factory reset app and wifi images in external flash had to remain untouched - this means we need to store up to two new wifi images in external flash.
  3. We wanted to be able to OTA across changes in the API between the module and the app - implying the need for a sync'd change in both with appropriate fail-safes.
  4. We use a cryptographic checksum to both label and verify wifi firmware image downloads (as we already did with our app images).


The required code changes separate into the following parts:

  1. Modify our app to be able to download wifi fw images into either App1 or App2.
  2. Add new fields to the end of our app DCT storage (the end being the only place we _could_ add them) so that our app knows what wifi image to use at module init time.
  3. Add host_platform_* hooks to WICED so that our app code could tell the WICED module init code where to get the wifi firmware image from
  4. Modify our app to update the APP LUT table the _first_ time we OTA'd to the new code to avoid fragmenting the App1 and App2 images.
  5. Modify the WICED code inside the wwd_management_wifi_platform_init() call tree to use the new hooks when loading wifi firmware onto the module.

We describe each in more detail in the corresponding sections below.

Download wifi fw images into either App1 or App2.

This was easy and doable entirely within our app using existing waf API's - no WICED changes needed.  We added logic to read the URL and download checksum for the wifi image along with the app image. And then all we had to change was the app index given to wiced_framework_app_open() during the download (i.e. DCT_APP1_INDEX or DCT_APP2_INDEX  instead of DCT_APP0_INDEX).

Add new fields to the end of our app DCT

Specifically, we added

  • Two integer fields keep track of which wifi firmware image  is active (identifies one of factory reset, App1, or App2) and which worked previously for recovery attempts when needed.
  • One integer field keeps track of how many times we've tried to init the wifi module using the currently selected image (used for recovery cases)
  • Three structure fields to keep track of the status info for each wifi image (size, validity, download checksum).

Your DCT additions will depend on how your app handles failure and recovery, but it will probably need a similar set of information.

Add new host_platform_* hooks to WICED  to select wifi firmware.

Inside WICED/WWD/include/platform/wwd_resource_interface.h

we added the following

{

WWD_RESOURCE_WLAN_FIRMWARE,    // used to refer to the wifi factory reset image

WWD_RESOURCE_WLAN_NVRAM,

WWD_RESOURCE_WLAN_FIRMWARE_A,  // used to refer to the wifi image stored in App1

WWD_RESOURCE_WLAN_FIRMWARE_B   // used to refer to the wifi image stored in App2

} wwd_resource_t;

   

extern uint8_t host_platform_wifi_get_firmware_resource(

wwd_resource_t *resource, uint32_t *file_size );   

extern uint8_t host_platform_resource_get_app_index(wwd_resource_t resource);

   

We included WEAK definitions of the default behavior for these two new functions directly inside WICED so that other apps/demo or apps/snip projects would still work as before. And then our app implemented overrides for the hooks using info from the new DCT fields to decide which wifi resource to load from and to manage recovery in failure cases.

Update the APP LUT on the first OTA to the new code

It was easy to detect that first run by checking that our new DCT fields were not yet initialized.  Since our production code previously only ever used App0, the free external flash could not yet be fragmented, so it was easy to fix the sizes of App0, App1, and App2 at the max we need for each using calls to wiced_framework_app_set_size().  This had the result of producing a fixed APP LUT table for external flash that looks like

AppLUT

Factory Reset wifi firmware image - stored as a *.bin image.

Factory Reset app  firmware image - stored as a stripped elf file.

App0 - OTA app update image

App1 - OTA wifi image A  - stored as a *.bin image.

App2 - OTA wifi image B  - stored as a *.bin image.

   

Modify WICED to use new hooks to select wifi firmware when loading the module.

For easy reference, the module initialization call tree looks like so

        wiced_init

            wwd_management_wifi_platform_init

                wwd_bus_init

                    wwd_bus_sdio_download_firmware

                        wwd_bus_write_wifi_firmware_image

                            download_resource

First, the original  wwd_bus_write_wifi_firmware_image() hardcodes the firmware resource to WWD_RESOURCE_WLAN_FIRMWARE.  So we changed it to use one of the new hooks, letting our app specify any of the three FR, A, or B wifi images:

WICED/WWD/internal/bus_protocols/wwd_bus_common.c::wwd_bus_write_wifi_firmware_image() {

wwd_resource_t wifi_resource;

host_platform_wifi_get_firmware_resource( &wifi_resource,... );

result = download_resource( wifi_resource ... );

}

Next, we had to modify two existing host_platform_resource* functions to replace the hard-codes of DCT_WIFI_FIRMWARE_INDEX to support the new image locations.  The mods were the same in both functions - here’s a snippet:

WICED/platform/MCU/wwd_resources.c

::host_platform_resource_size()

::host_platform_resource_read_direct()

if ( resource == WWD_RESOURCE_WLAN_FIRMWARE    ||

     resource == WWD_RESOURCE_WLAN_FIRMWARE_A  ||

     resource == WWD_RESOURCE_WLAN_FIRMWARE_B  )

{   ...

   #ifdef WIFI_FIRMWARE_IN_MULTI_APP

   uint8_t app_index = host_platform_resource_get_app_index(resource); // new hook

    ...

    if ( wiced_waf_app_open( app_index ...) != WICED_SUCCESS ) // replace hard-code

    ...

You'll want your app logic for the OTA process to choose the "most available" wifi image to download into, mark it as invalid in the DCT right before copying into it, and then only mark it as valid if the download checksum passes.  For our implementation, we would fail the entire OTA if the wifi image download failed, and would only download the app image if the wifi image download succeeded. We would only update the DCT tracking of the "active" wifi image if both downloads succeeded and we commanded a reboot to the new app.  This kept app and wifi image in sync. And our implementation of the host_platform_wifi_get_firmware_resource() hook included logic to recover if a new wifi image failed to load or init properly after a several reboot tries, at which point we would revert to the older wifi image.

If you have space in your external flash for two complete wifi image binaries, then these relatively simple changes should be all you need.  I would estimate it only took us about 3 days to implement and test this much.

Compressed Wifi image version

Unfortunately for us, we could only fit two new wifi images into our external flash if we compressed them by about 15% first. Conceptually, this isn't hard.  We just needed to compress the binaries before uploading to our server and then add the required streaming decompression to our version of download_resource.

This did add the need for an extra check that we could successfully decode the image we download. To handle this, we added a second crypto checksum for the decoded image.   So our OTA logic now downloads the compressed wifi image into external flash, verifies the download checksum during the transfer, and then we actually immediately decode the image from external flash and verify that decoded checksum also matches.  We do this before proceeding to the app image download. If we fail either checksum here then we consider the wifi download failed and, as in the basic case, we fail the entire OTA and retain the original app and wifi image.

We chose a simple lossless Lempel-Ziv (lz77) encoding because a streaming decoder for lz77 is only about 400 words.  But as a result, for every fixed-size chunk of encoded data we would read out of external flash inside download_resources(), we would generate a variable sized decoded chunk.  This is where we ran into two "gotchas" in the WICED guts.  

Gotcha #1

To understand the issues, we have to look at the "unmodified" download_resources() function.  Here's a condensed version:

WICED/WWD/internal/bus_protocols/wwd_bus_common.c::download_resource(){

host_platform_resource_size(resource, &size)

while (transfer_progress < size) {

  host_platform_resource_read_indirect(..., &segment_size)

  for(segment_size != 0; segment_size-=transfer_size,*address += transfer_size...) {

    transfer_size = MIN( WWD_BUS_MAX_TRANSFER_SIZE, segment_size ); // GOTCHA #1!

    wwd_bus_set_backplane_window(address)                                            

    wwd_bus_transfer_bytes(... address & BACKPLANE_ADDRESS_MASK, transfer_size, ...)

  }

}}

As you can see, the function gets the size of the resource from host_platform_resource_size(). It then reads the image, one fixed_size block at a time, and copies that block to the module over the bus (SDIO in our case).   Each block is exactly WWD_BUS_MAX_TRANSFER_SIZE up until the very last block, which is just the remainder. Here is gotcha #1 if you're not careful. The code inside the for loop is assuming that it can safely copy all transfer_size bytes with only ONE setting of the backplane window. That works fine IF each segment_size input to the for-loop is aligned to a multiple of the backplane window size.  

But as soon as we add the decompression we need, right after the host_platform_resource_read_indirect() call, we now have basically arbitrary segment_sizes.  In that case, eventually the code will hit a situation where the address at the start of the for loop is near the top of the backplane window and the transfer_size copy would overflow the window, resulting in some of the image NOT being copied to the module.

   

There are several ways to fix this, but we wanted to minimize buffer sizes, so it was sufficient to have the for-loop ensure that it can complete its copy safely within each iteration.  We do this by calculating how much space is available inside the backplane window given the address at the start of the for-loop and then modify transfer_size accordingly if necessary. Here’s a condensed version of that code:

download_resource () {

host_platform_resource_size(resource, &size)

while (transfer_progress < size) {

  host_platform_resource_read_indirect(...)

  <do decompression here>

  for ( ; segment_size != 0; ...) {

    offset = *address & BACKPLANE_ADDRESS_MASK;

    space = (BACKPLANE_ADDRESS_MASK - offset + 1);

    transfer_size = MIN( WWD_BUS_MAX_TRANSFER_SIZE, segment_size );

    transfer_size = (uint16_t) MIN( transfer_size, space ); // Ensure copy fits

           

    wwd_bus_set_backplane_window(address)

    wwd_bus_transfer_bytes(... offset, transfer_size, ...)

  }

}}

Gotcha #2

Once we got past gotcha #1, we ran into another issue exposed by the variable transfer_sizes going into wwd_bus_transfer_bytes().

Namely, every transfer except the last one needed to have a transfer_size that was an integer multiple of 4 (i.e. an integer number of 32-bit words) or the module firmware load would timeout. This is gotcha #2.

On closer inspection, the functions  inside wwd_bus_transfer_bytes use SDIO_BLOCK_MODE for any sizes above 63 bytes.  The implementation of that mode for the STM32F4xx micro inside

WICED/platform/MCU/STM32F4xx/WWD/sdio_prepare_data_transfer()

uses DMA, which is restricted to word transfers.  So even though wwd_bus_transfer_bytes is named as if it copies an arbitrary number of bytes across the bus, it actually can only transfer words.

Here's the unmodified call tree snippet for our platform:

WICED/WWD/internal/bus_protocols/SDIO/wwd_bus_protocol.c
::wwd_bus_transfer_bytes() {

  wwd_bus_sdio_transfer

  host_platform_sdio_transfer()

    sdio_prepare_data_transfer() {

      ...

      DMA2_Stream3->NDTR = dma_transfer_size/4;// Convert num bytes to num words.

}

where host_platform_sdio_transfer() and subsequent calls are inside

WICED/platform/MCU/STM32F4xx/WWD/wwd_SDIO.c.

In principle, you could modify wwd_bus_transfer_bytes() to use a combination of SDIO_BYTE_MODE and SDIO_BLOCK_MODE copies to actually copy an arbitrary byte size. 

In practice, we simply modified the download_resource code to only copy to the bus in integer multiples of 4 bytes up until the last copy.  Any decoded output remainder we had left (0..3 bytes) at the end of a while loop iteration we would simply copy into the beginning of the input buffer for the next iteration to handle.  That made for some annoying bookkeeping, but was still relatively straightforward. And with that dodge in place, our solution works reliably.

Ralph Sherriff wrote:

On closer inspection, the functions  inside wwd_bus_transfer_bytes use SDIO_BLOCK_MODE for any sizes above 63 bytes.  The implementation of that mode for the STM32F4xx micro inside

WICED/platform/MCU/STM32F4xx/WWD/sdio_prepare_data_transfer()

uses DMA, which is restricted to word transfers.  So even though wwd_bus_transfer_bytes is named as if it copies an arbitrary number of bytes across the bus, it actually can only transfer words.

To cypress,

This needs to be fixed in the low level code to allow copy any arbitrary number of bytes across the bus.

Don't workaround it in the callers especially you cannot make sure if the binary library code hit such issue.

0 Likes
Anonymous
Not applicable

Hi RSH,  What is your host controller? Is it MCU or Linux base mother board? what is your interface USART or SPI?

Did you transfer big data like few MB to Host?

0 Likes

Our host is an STM32F4 running the ThreadX RTOS and interfaced to the 4343W via SDIO.

Our application does not typically involve large data transfers.  The OTA images are the largest things we move.

0 Likes

Hi RSH,

This information is very useful because we would like to configure OTA function to update not only main app but also Wi-Fi FW/CLM_BLOB with STM32F412+CYW43438.

My understanding in default ota.fr.c is follows.

1. Get main app by uploading from external PC through Wi-Fi.( We can change this potion to download FW from the internet server, like you)

2. Gotten new main app will be stored external flash, temporary.

3. over write main app from external flash(new one) to internal flash(existing one).

Following is my question.

Q1: Is my understanding of default WICED OTA(not OTA2) is correct?

Q2: If so, which function in FW will do #3 portion(over-write to internal flash) above ?(Bootloader? or other code?)

Q3. Do you know how to implement to update CLM_BLOB?(We have a plan to use latest WICED)

Q4: If you can, could you provide your sample application that achieve your solution?(We can ready FTP server) ?

Currently, I'm now deepdive WICED code but still not familiar with it yet.

Thank you.

--TAK

0 Likes

BTW, Cypress said that

-STM32F412 can not execute main app from external flash.

-This is specified by STM specification.

Do you think it that true?

0 Likes