4 Replies Latest reply on Jun 13, 2018 5:13 PM by user_474925260

    Over the air firmware updates working but not 100% of the time... and that's a problem.

    user_474925260

      Ok, so I have had the updatable stack OTA working from an iPhone app for the past few months.  I can save the 'target firmware' (stack and app) in the iOS phone build and if the target is determined to be out of date, then the phone will command the cypress to reboot into OTA/Bootloader mode and the update will proceed as expected.

       

      Except for a few times in ~100 where the unit will reboot as commanded but continues to come back up in the App mode instead of OTA...  This is rather critical because if it will only reboot back into the app mode, then it can never update the firmware and the feature suddenly doesn't exist for that unit.  If I reprogram the device again with just the bootloader and stack, then it will work again for a while... but then later it stops rebooting into OTA mode...

       

      So...  ==> Is there any known issue with the PSoC/Cypress part not rebooting correctly back to OTA mode?  This is using standard PSoC/Cypress code to accomplish the reboot.

       

      Here are the notes I took as I was trying to figure this out... I sent them to someone else to explain the issue so it reads that way...

       

      ****

       

      Ok, here is enough of the log to see the looping as printed out from the Cypress over UART.

       

       

      It decides to reboot to OTA, starts printing the fact that it should and then boots back into normal mode (I can tell because of the FWDID that is only printed in normal mode - not OTA mode)…. Phone sees date is wrong and commands it to reboot again, etc…

       

       

      [This is the output from the Cypress UART; the CYP: is prepended by my UART capture app.]

       

       

            CYP: !Rebooting to OTA

            CYP: !Bootloadab !FWSERN 0

            CYP: !FWBOOT Dec 30 2017 16:39:17

            CYP: !FWSTCK Jan 26 2018 07:57:21

            CYP: !FWAPPL Feb  3 2018 11:32:10

            CYP: !FWDID 0x1946de18 0x000b1a2b

            CYP: ?DT4

            CYP: ?!STARTBLE

            CYP: ?DT4

            CYP: ?

            CYP: !FWADR -85 0x00a0503196ab

            CYP: ! !Connected

            CYP: +BLE SN: 12 ID: pogoGS      UID: 18604204

            CYP: ~BATT 5139

            CYP: !Rebooting to OTA

            CYP: !Bootloadab !FWSERN 0

            CYP: !FWBOOT Dec 30 2017 16:39:17

            CYP: !FWSTCK Jan 26 2018 07:57:21

            CYP: !FWAPPL Feb  3 2018 11:32:10

            CYP: !FWDID 0x1946de18 0x000b1a2b

            CYP: ?DT4

            CYP: ?!STARTBLE

            CYP: ?DT4

            CYP: ?

            CYP: !FWADR -85 0x00a0503196ab

            CYP: ! !Connected

            CYP: +BLE SN: 12 ID: pogoGS      UID: 18604204

            CYP: ~BATT 5143

            CYP: !Rebooting to OTA

            CYP: !Bootloadab !FWSERN 0

            CYP: !FWBOOT Dec 30 2017 16:39:17

            CYP: !FWSTCK Jan 26 2018 07:57:21

            CYP: !FWAPPL Feb  3 2018 11:32:10

            CYP: !FWDID 0x1946de18 0x000b1a2b

            CYP: ?DT4

            CYP: ?!STARTBLE

            CYP: ?DT4

            CYP: ?

            ...etc

       

       

       

       

      Here is the relevant cypress code.  The Set active App is supposed to set something in flash (or ram) so it knows to boot to BL0 (bootload / OTA).  The Bootloadable_Load() actually does it’s own CySoftwareReset so that is why the !Bootladable_Load UART above gets truncated (it rebooted before it finished sending the UART) and also why we don’t see the CySoftwareReset printout happening above.

       

       

                // or we can command it to reset into bootloader mode

       

       

            else if (wrReq->handleValPair.value.val[1] == 0x38)

            {

                UART_UartPutString("!Rebooting to OTA\r\n");

       

       

                if (Bootloadable_SetActiveApplication(0) == CYRET_SUCCESS) {

                    UART_UartPutString("!Bootloadable_Load\r\n");

                    Bootloadable_Load();

                    UART_UartPutString("!CySoftwareReset\r\n");

                    CySoftwareReset();

                }

                else

                    UART_UartPutString("!Failed to SetActiveApplication\r\n");

            }

       

       

       

       

      The bootloadable load looks like this (so pretty brief) [this is PSoC generated code]

       

          /*******************************************************************************

          * Function Name: Bootloadable_Load

          ****************************************************************************//**

          *

          * \brief

          *  Schedules the Bootloader/Launcher to be launched and then performs

          *  a software reset to launch it

          *

          * \return

          *  This method will never return. It will load a new application and reset

          *  the device.

          *

          *******************************************************************************/

          void Bootloadable_Load(void)

          {

              /* Schedule Bootloader to start after reset */

              Bootloadable_SET_RUN_TYPE(Bootloadable_SCHEDULE_BTLDR);

              CySoftwareReset();

          }

       

      The set run type is: [this is PSoC generated code]

       

          /*******************************************************************************

          * Schedule the Bootloader/Bootloadable to be run after a software reset.

          *******************************************************************************/

          #if(CY_PSOC4)

              #define Bootloadable_SET_RUN_TYPE(x)        (cyBtldrRunType = (x))

          #else

              #define Bootloadable_SET_RUN_TYPE(x)        CY_SET_REG8(CYREG_RESET_SR0, (x))

          #endif  /* (CY_PSOC4) */

       

      The cyBtldrRunType is just a location in flash (or ram…?) [this is PSoC generated code]

       

       

          #if (CYDEV_PROJ_TYPE != CYDEV_PROJ_TYPE_STANDARD)

              /*******************************************************************************

              This variable is used by the Bootloader/Bootloadable components to schedule

              what application will be started after a software reset.

              *******************************************************************************/

              #if (__ARMCC_VERSION)

                  __attribute__ ((section(".bootloaderruntype"), zero_init))

              #elif defined (__GNUC__)

                  __attribute__ ((section(".bootloaderruntype")))

              #elif defined (__ICCARM__)

                  #pragma location=".bootloaderruntype"

              #endif /* (__ARMCC_VERSION) */

              volatile uint32 cyBtldrRunType;

          #endif  /* (CYDEV_PROJ_TYPE != CYDEV_PROJ_TYPE_STANDARD) */

       

      The memory map of that location across the BOOT / STACK / APP looks like this.  The 0x2000000 is basically just at 512K so this is past flash area (unless they map it differently but the .ramvectors sounds like ram…)

       

       

      BOOT:

       

          .ramvectors     0x20000000       0xc0

                          0x20000000                __cy_region_start_ram = .

          *(.ramvectors)

          .ramvectors    0x20000000       0xc0 .\CortexM0\ARM_GCC_541\Debug\Cm0Start.o

                          0x20000000                CyRamVectors

          .btldr_run      0x200000c0        0x4

          *(.bootloaderruntype)

          .bootloaderruntype

                          0x200000c0        0x4 .\CortexM0\ARM_GCC_541\Debug\Cm0Start.o

                          0x200000c0                cyBtldrRunType

          .noinit

          *(.noinit)

          .data           0x200000c8       0xa0 load address 0x00001628

                          0x200000c8                __cy_region_start_data = .

                        

      STACK:

       

          *(.ramvectors)

          .ramvectors    0x20000000       0xc0 .\CortexM0\ARM_GCC_541\Debug\Cm0Start.o

                          0x20000000                CyRamVectors

          .btldr_run      0x200000c0        0x4

          *(.bootloaderruntype)

          .bootloaderruntype

                          0x200000c0        0x4 .\CortexM0\ARM_GCC_541\Debug\Cm0Start.o

                          0x200000c0                cyBtldrRunType

          .noinit

          *(.noinit)

          .data           0x200000c8      0x640 load address 0x0001fb98

                          0x200000c8                __cy_region_start_data = .

       

       

      APPL:

          .ramvectors     0x20000000       0xc0

                          0x20000000                __cy_region_start_ram = .

          *(.ramvectors)

          .ramvectors    0x20000000       0xc0 .\CortexM0\ARM_GCC_541\Debug\Cm0Start.o

                          0x20000000                CyRamVectors

          .btldr_run      0x200000c0        0x4

          *(.bootloaderruntype)

          .bootloaderruntype

                          0x200000c0        0x4 .\CortexM0\ARM_GCC_541\Debug\Cm0Start.o

                          0x200000c0                cyBtldrRunType

                          0x00001938                BOOTLOADER_RAM_SIZE = (Bootloader___bss_end__ - 0x20000000)

          .bootloader_data

                          0x200000c8     0x1938

                          0x20001a00                . = (. + BOOTLOADER_RAM_SIZE)

          *fill*         0x200000c8     0x1938

          .noinit         0x20001a00     0x1034

          *(.noinit)

          .noinit        0x20001a00        0x4 .\CortexM0\ARM_GCC_541\Debug\Cm0Start.o

          *fill*         0x20001a04        0x4

          .noinit        0x20001a08     0x102c .\CortexM0\ARM_GCC_541\Debug\FW-TRACKER.a(fwGS.o)

                          0x20001a08                cyBle_stackMemoryRam

          .data           0x20002a38      0x920 load address 0x0002b408

                          0x20002a38                __cy_region_start_data = .

       

       

      Only oddity I see here is the bootloader ram size stuff before the load address but maybe that is normal.

       

       

      Is there any better way to force this reboot into a certain code space?  Is it possible to just JMP to the start of OTA/Stack code?  Since I build the App after the Stack, it would be possible for the App to know the start location of the Stack code (or seems like it anyway)...
      Thank you for your responses!
      /kjs
        • 1. Re: Over the air firmware updates working but not 100% of the time... and that's a problem.
          Gyan Chand

          Hello,

           

             So if I summarize your description," You are sending a command 0x38 , from Smart Phone to the Cypress BLE device to put the Cypress BLE device in Bootloader Mode and receive a new image but sometimes, even thought the Cypress device switches to the Bootloader Mode on the command receive but it comes back to Application instantaneously " .Am I right??

           

          Could you please check what is the 'wait for command' time set for your bootloader project? ( In stack Project-> Double Click on Bootloader Component -> Wait for Command)

           

          Once the device switches to Bootloader Mode ,it waits for the time specified in this field and if there is no command received from the host(Smart Phone) in this time interval then the device back switches to Application.

           

           

          Regards,

          Gyan

          1 of 1 people found this helpful
          • 2. Re: Over the air firmware updates working but not 100% of the time... and that's a problem.
            user_474925260

            Gyan,  You are almost correct on my description but I don't think it is even entering the bootloader mode at all but rather just resetting back to the application.  I say this because my OTA_Stack app sends some minimal info to the UART as does my Application.  When it resets, I only see the UART output repeated for the Application as if it skipped the OTA/Bootloader altogether.  Oh, I am doing the upgradable stack and have successfully before upgraded both stack and application.

             

            in answer to your question, the wait for command is set to 2000 (2 seconds). 

             

            When the phone sends the 0x38, it immediately then tries to reconnect.  I found that the UUID changed from stack to application, so I actually have to do a BLE scan with the phone and detect the device that I know I just commanded to reset.  [I have the advertising packet set different for the Stack/Bootloader vs the App but have a serial number in it so I can be sure I'm reconnecting to the one I just disconnected from.].

             

            Please let me know if you can think of anything else.  I still have that unit here and last time I had one in this state and tried to do the 'attach debugger', I ended up just reprogramming it.

             

            Here is the UART where the highlighted part is back in the application with no stack UART happening.  I'll go see if I can generate what a normal display looks like and edit this post with that.

             

                  CYP: !Rebooting to OTA

                  CYP: !Bootloadab !FWSERN 0

                  CYP: !FWBOOT Dec 30 2017 16:39:17

                  CYP: !FWSTCK Jan 26 2018 07:57:21

                  CYP: !FWAPPL Feb  3 2018 11:32:10

                  CYP: !FWDID 0x1946de18 0x000b1a2b

                  CYP: ?DT4

                  CYP: ?!STARTBLE

                  CYP: ?DT4

             

            Thank you!

            • 3. Re: Over the air firmware updates working but not 100% of the time... and that's a problem.
              user_474925260

              Gyan,

               

              This here is the normal UART display when the reboot to bootloader is successful and then the reboot back to application after the OTA update:

               

              !Rebooting to OTA

              !Bootloadab

              !FWSERN 0

              !FWBOOT Dec 30 2017 16:39:17

              !FWSTCK Jan 26 2018 07:57:21

              !FWAPPL N/A <=== this tells me it has succesfully switched to OTA/Bootloader mode because the app date/time is now N/A

              +OTA SN: 0 ID: FthrWtTRAKR UID: 246360880

              !BLSN: 0

              !FWDID 0x1946de18 0x000b2122

              !FWADR 0x00a050000614

              +OTA SN: 0 ID: FthrWtTRAKR UID: 246360880

              !Send FW0 #

              !Send FW1 #

              !Send FW2 #

              <== OTA update is happening here ==>

              !FWSTCK Jan 26 2018 07:57:21

              !FWAPPL N/A

              !FWSERN 0

              !FWBOOT Dec 30 2017 16:39:17

              !FWAUP <== this is the app detecting that it was updated

              !CCCD number has not changed.

              !FWSTCK Jan 26 2018 07:57:21

              !FWAPPL Feb 13 2018 17:48:25 <== this is the new build date/time for the application

              !FWDID 0x1946de18 0x000b2122

              • 4. Re: Over the air firmware updates working but not 100% of the time... and that's a problem.
                user_474925260

                After some number of months of having this intermittent bootloader problem, and after having the units 'brick' themselves occasionally, I found that the bootloader writes to two rows of flash every time it boots.  if you have poor power during that process, it corrupts those rows and fails to boot to either stack or application after that.  this is in the bootloader code from psoc.... maybe it has been updated since the build I am using...  I fixed it by updating it to not update the flash if the bytes didn't change.  [we could easily brick a device by just connecting ground and tapping power to simulate a bad connection and/or poor battery.]

                 

                I was going to go post the solution as a separate thread but now found this which is basically the same thing:

                 

                OTA upgradeable stack corruption of metadata

                 

                [for this thread, I think it managed to corrupt the row for the STACK so it always rebooted back to the APP even though I was trying to boot to STACK to do an update]

                1 of 1 people found this helpful