9 Replies Latest reply on Oct 2, 2014 6:25 AM by legic_1490776

    BCM20732S based boards sometimes develop UART failure

    legic_1490776

      I had previously posted a question related to this problem here:

      BCM20732S refuses to program

      but I thought I would start a new thread since the problem may not be related to the EEPROM.

       

      I have a design based on the 20732S that overall seems to work quite well.  However, a few boards develop what appears to be a hardware problem with the UART after some period of time in use.  Note that these boards are being used for development, so we reprogram them via the UART frequently (whereas in the application this will only happen once).

       

      What we observe is that we will have a board that is working fine for a long time, and then at some point I try to program it via the UART and the board is no longer detected.  From that point onwards, I can never get it to program again.  There doesn't seem to be any particular event that triggers this, although in one case it happened after the serial line was accidentally disconnected during programming.

       

      In trying to diagnose the problem I put a scope on the RX and TX lines of the UART, and I can see why the chip fails to detect.

       

      Looking at the HCI serial port with a scope, I see unusual spikes on the TX line when the chip is transmitting.  That is, I reset the chip with HCI_RX held high to bring it into the bootloader.  Then, I use the IDE to detect and program.  The IDE first goes through the detection process, and attempts to contact the boot loader over the HCI port by sending a string  ‘ 01 03 0c 00 01 4d fc 05 1c d2 00 00 01 ‘. The chip responds with another string, but the response is garbled because of strange spikes that are occurring at exactly 1/10 the baud rate, at 11.52 kHz.

       

      These spikes can be seen in the attached photo of the scope trace. You can see the blue trace is showing a bit on the TX line -- the chip is trying to respond to the detection string.  However interspersed with the real bits are weird ramping up and spiking glitches.  These are occurring at exactly 11.52kHz.

       

      We are concerned about this problem because we are moving toward production and as yet we don't understand why this happens or what causes it.  It might be OK if it is related to programming failure that only happens in development when we are reprogramming a lot from HCI (vs once in production)... but at this point we are nervous.

       

      One more point to consider: we have found that we needed to add the PMU clock warmup time hack to get our boards to work, is it possible that this problem is somehow related to that?

       

      I tried heating up the chip with a heat gun to see if I could program it, but this did not help.  I have also attempted to hold down SDA on boot to try to reset the EEPROM, which did not work.  I was doubtful that it would help, since the problem seems to be with the UART / bootloader.

       

      So far on our development boards, this has happened on about 10% of the boards overall, which is a pretty high rate.

        • 1. Re: BCM20732S based boards sometimes develop UART failure
          jota_1939431

          Hello Idgirod,

           

          Would like to get more details on the 10% failure rate:

           

          Do you have a Pull-down or Pull-up Resistor on the TX line?  That might be the reason for the glitch on the scope.

          The glitch seems to be the same voltage ramp on the image attached.  Do you have a RC network attached to the TX line?

           

          Thanks

          JT

          • 2. Re: BCM20732S based boards sometimes develop UART failure
            legic_1490776

            HI JT,

             

            There is no pullup or pulldown on the TX line, or RC network. 

            The TX line is a completely clean, and I have it plugged directly into a standard "FTDI" 1.8v TTL serial-USB cable.

             

            Also note that initially the boards do not exhibit this strange ramp/glitch; the serial looks completely normal on working boards.

             

            Then, something happens (the chip breaks in some way), after which these strange glitches are present.

             

            Re. the 10% failure rate:

            We have built 5 revisions of this board, and at least one board from each revision has exhibited this problem. 

            Revs 4 and 5 are more or less final and otherwise seem to work well, but out of the 17 of these we have tested with, 3 of the boards have "bricked" in this way.

             

            In all cases the failed board had previously worked correctly, in some cases for quite a long time, before failing.

            In at least one case the serial line was accidentally disconnecting during programming, directly prior to the failure.

            • 3. Re: BCM20732S based boards sometimes develop UART failure
              jota_1939431

              Hello idgirod,

               

              What is the VDDIO voltage level that you are programming the device?

               

              We usually use the FTDI Cable below for our programming.

               

              For your Standard "FDTI" cable, which one are you using - the TTL_232R-3V3 or the TTL_232R-3V3-WE:

               

              Please make sure that you are using the 3.3V Signaling cable and not the 5V Cable.

               

              If you are using the 1.8V cables, what is your VDDIO rail.

               

              I have made this mistake before.

               

              Please let us know

               

              Thank you,

               

              JT

              • 4. Re: BCM20732S based boards sometimes develop UART failure
                legic_1490776

                Hi JT,

                 

                Our system runs everything off a 1.8V rail, and we are using the 1.8V wire-ended FTDI cable.

                We power the board from a coin cell at all times (with everything running off the 1.8V regulator).

                We don’t use the USB cable for power, only RX,TX and GND connected, with RX/TX at 1.8V.

                 

                This normally works without problems, until a board develops this problem after a while.

                • 5. Re: BCM20732S based boards sometimes develop UART failure
                  jota_1939431

                  Hello idgirod,

                   

                  I discussed this with our team and you should have no issue programming in this fashion.

                   

                  Questions: Our SDK verifies after a download.  Have you disabled this feature?

                  If you deviate from the 1.8V source, that may damage the IO that would brick the devices.

                   

                  Let us know your feedback.

                   

                  Thanks

                  JT

                  • 6. Re: BCM20732S based boards sometimes develop UART failure
                    ShawnA_01

                    Hi ldgirod,

                     

                    1. Can you send lotcodes for the failing modules?

                    2. How long have you had these parts in-house?

                    3. You mention the boards operate from coin-cell plus the UART (via FDTI) is running at 1.8V. However, the waveforms show UART running at 3.3v.   Are these old pictures from some previous board?

                    4. Are you always using an FDTI (USB<->UART) bridge device, or do you have another mechanism?

                    • 7. Re: BCM20732S based boards sometimes develop UART failure
                      legic_1490776

                      Answers:

                       

                      1. The affected lot codes are TN1337 and TN1351.  Most of our boards have TN1337, but we have seen the failure in both.

                       

                      2. I believe that we purchased the parts in february of this year.  We assembled the last of our boards with these parts in June.

                       

                      3. Oops.. sorry about the confusing photo.  That photo was from a previous revision that did run everything off 3.3V and at that time we were using the 3.3V version of the FTDI cable.  The current board uses 1.8V and a 1.8V cable.  I've attached updated photos.

                       

                      4. yes, we always use the FTDI cable.  Some previous versions of the board had an on-board USB converter, but the last two revisions had only had a 3 pin connector to the FTDI cable (gnd, rx, tx.)

                       

                      Here are some updated photos:

                      RX.jpg

                      The RX line (PC->chip) showing the probe request used during device detection from SDK. Note serial is at 1.8V

                      normalTX.jpg

                      The TX line (chip->PC) for a normal chip, showing the response from the boot loader.

                       

                      brokenTX.jpg

                      TX output for a broken chip.

                      brokenTXzoom1.jpg

                      zoomed in to broken TX response, you can see the spikes.  Also note the strange glitches before the chip tries to send data.

                       

                      brokenTXzoom3.jpg

                      Zoomed in more to see the glitch waveform has a strange shape.

                      • 8. Re: BCM20732S based boards sometimes develop UART failure
                        legic_1490776

                        One more thing.. we've seen this failure in all 5 revisions of the board so far.  Earlier revisions had other problems which we eliminated with design changes, but this problem remains.

                        • 9. Re: BCM20732S based boards sometimes develop UART failure
                          legic_1490776

                          OK, I have finally found a bizarre solution which appears to allow me to recover devices which have developed this problem.

                           

                          The solution is to apply power to the device WITH THE UART DISCONNECTED.

                          Then, connect the UART.  Do not reset the device after connecting the UART, as you would do normally.

                           

                          At this point, as long as it has not been reset with the UART connected, the device will detect and program properly.

                           

                          Note that this method will NOT program a device that has a valid program installed, as the already loaded application will start immediately if the UART is not connected on powerup / reset / boot.

                           

                          However this appears to be the only way to bring back a device that has developed this problem.

                           

                          I am still not sure what causes this condition, but it appears to be related to interrupting the programming process.