9 Replies Latest reply on Sep 23, 2020 5:50 AM by MaCa_2922036

    WICED 101 httpbin_post_tls

    RoMc_863046

      Hello,

      I am able to compile and get a response from httpbin.org using the example in WICED 101 07b_06.  This is the project titled httpbin_post_tls.

      I have modified the example to send a JSON sting to MY server instead of httpbin.  I have put the necessary certificates for my server in place.  I have verified with the server admin that my POST has been received and the server is sending the appropriate response, however, I am not receiving the payload in WICED.  I do receive the HTTP response header, just no payload (body) data.

       

      I have this working with a different module manufacturer and I compared the HTTP response headers.  They are identical.

      Is there an additional step I need to take in order to receive the payload?

       

      The HTTP response indicates that it is chunked:

       

      HTTP/1.1 200 OK

      Server: nginx/1.14.0 (Ubuntu)

      Date: Fri, 11 Sep 2020 14:36:56 GMT

      Content-Type: application/json

      Transfer-Encoding: chunked

      Connection: keep-alive

      Expires: Thu, 19 Nov 1981 08:52:00 GMT

      Access-Control-Allow-Credentials: true

      Access-Control-Allow-Origin:

      Access-Control-Allow-Headers: X-Requested-With, Content-Type, Accept, Origin, Authorization

      Access-Control-Allow-Methods: GET, POST, PUT, PATCH, DELETE, OPTIONS

      Cache-Control: no-store, no-cache, must-revalidate, max-age=0, post-check=0, pre-check=0

      Pragma: no-cache

      Set-Cookie: PHPSESSID=5e2ojh23ud2fc09kn63hrrcdua; path=/

      Strict-Transport-Security: max-age=31536000

      Referrer-Policy: same-origin

      X-Frame-Options: SAMEORIGIN

      X-XSS-Protection: 1; mode=block

      X-Content-Type-Options: nosniff

      X-Frame-Options: SAMEORIGIN

       

      Thank you,

      -Rocko

        • 1. Re: WICED 101 httpbin_post_tls
          ChunleiL_51

          Hi,

           

          I think this is the code example you are using.

          CypressAcademy_WW101_Files/Projects/ww101key/07b/06_httpbin_post_tls at master · cypresssemiconductorco/CypressAcademy_W…

           

          To help with the debugging, please provide the test result that you run this example without modification on the same board, to see if it can get the expected payload from httpbin.org.

          This step can help you analyze which side the problem exists in, the server or the client.

           

          Best regards

          • 2. Re: WICED 101 httpbin_post_tls
            RoMc_863046

            Hello ChunleiL_51,

             

            Here is the output from httbin_post_tls with httpbin.org payload.

             

            Starting WICED vWiced_006.002.001.0002

            Platform LAIRD_EWB initialised

            Started ThreadX v5.8

            Initialising NetX v5.10_sp3

            Creating Packet pools

            WLAN MAC Address : C0:EE:40:70:35:61

            WLAN Firmware    : wl0: Apr 30 2018 04:14:19 version 7.45.98.50 (r688715 CY) FWID 01-283fcdb9

            WLAN CLM         : API: 12.2 Data: 9.10.158 Compiler: 1.29.4 ClmImport: 1.36.3 Creation: 2019-05-07 05:05:14

            Joining : bacon

            Successfully joined : bacon

            Obtaining IPv4 address via DHCP

            DHCP CLIENT hostname WICED IP

            IPv4 network ready IP: 192.168.1.155

            Resolving IP address of httpbin.org

            httpbin.org is at 3.221.81.55

            Root Certificate Initialized

            Connected to httpbin.org

            ------------------ Received response: 1 ------------------

            ----- Response Header: -----

            HTTP/1.1 200 OK

            Date: Mon, 14 Sep 2020 13:40:47 GMT

            Content-Type: application/json

            Content-Length: 371

            Connection: keep-alive

            Server: gunicorn/19.9.0

            Access-Control-Allow-Origin: *

            Access-Control-Allow-Credentials: true

            ----- Response Payload: -----

            ------------------ Received response: 1 ------------------

             

             

            ----- Response Payload: -----

            {

              "args": {},

              "data": "{\"WICED\":\"yes\"}",

              "files": {},

              "form": {},

              "headers": {

                "Content-Length": "15",

                "Content-Type": "application/json",

                "Host": "httpbin.org",

                "X-Amzn-Trace-Id": "Root=1-5f5f72df-1c33ba8001dbf6da89028885"

              },

              "json": {

                "WICED": "yes"

              },

              "origin": "47.4.209.115",

              "url": "https://httpbin.org/post"

            }

             

             

            ------------------ End Response 1 ------------------

             

             

            -Rocko

            • 3. Re: WICED 101 httpbin_post_tls
              GrCa_1363456

              ChunleiL,

               

              I initiated a MyCase to allow the sharing of Wireshark captures that may contain sensitive and competitive information.

              Please reply to me outside of the Community if I can provide additional information or support.

               

              When this issue is resolved, I trust we can summarize the resolution back here in this Community Post to help others who may experience a similar issue.

               

              Greg C

              • 4. Re: WICED 101 httpbin_post_tls
                VaSt_4778721

                Hi Rocko,

                 

                Just a hint, maybe it helps...

                If you print the response->payload_data_length, it might indicate that it's 0.

                However if you do for example a:

                      print_content( (char*) response->payload, 10);      // 10 = magic number

                ...you'll notice the payload is present.

                 

                If this is indeed the case, you can consider creating a function that parses the header - similar to http_parse_header - which also looks for "Content-Length:" inside and returns the length of the body.

                 

                Please disregard my post if I'm completely off the track.

                 

                Good luck,

                V

                3 of 3 people found this helpful
                • 5. Re: WICED 101 httpbin_post_tls
                  VaSt_4778721

                     Extracting the body length from the header:

                   

                                  uint16_t value;

                                      char *ptr = response->response_hdr;

                                      while (ptr) {

                                          ptr = strstr(ptr, "Content-Length: ");

                                          if (ptr == NULL) {

                                              break;

                                          }

                                          ptr+=sizeof("Content-Length: ")-1;

                                          value = strtol(ptr, &ptr, 10);

                                          printf("%lu\n", value);

                                          continue;

                                      }

                   

                  RoMc_863046

                  1 of 1 people found this helpful
                  • 6. Re: WICED 101 httpbin_post_tls
                    RoMc_863046

                    Thank you both for your feedback.

                     

                    V,  You are correct.  When I print the body data with a "magic number" my payload IS there.  This is great news!

                     

                    Unfortunately, The header does not have a "Content-Length" field.  My full response header is in the original post.

                    I am in the process of migrating from a non Cypress WiFi module.  When using the other module, I leveraged the nodejs http-parser library.  That parser was able to determine payload length without a "Content-Length" field.  I will have to determine a different method for finding the content length.  This field is not always a guarantee, especially as servers migrate to HTTP/2.

                     

                    Thanks again.  If you have any ideas to help out with finding the payload length without the Content-Length field, other than bringing in the nodejs http-parser, please let me know.

                     

                    -Rocko

                     

                    VaSt_4778721

                    ChunleiL_51

                    • 7. Re: WICED 101 httpbin_post_tls
                      MichaelF_56

                      Appreciate the help VaSt_4778721

                      • 8. Re: WICED 101 httpbin_post_tls
                        ChunleiL_51

                        Hi RoMc_863046

                         

                        I finally reached to the conclusion that the issue resides in the WICED APIs. And I'm using WICED Studio 6.4.

                        Line 513 In the file <WICED_SDK>\43xxx_Wi-Fi\libraries\protocols\HTTP_client\http_client.c (i.e. the file that implement HTTP Client features) explains all. See below:

                        Since your server enabled Chunked Encoding, the payload of the response data should be handled in the IF block. But we can see there is no implementation in that block. Only a TODO is left. Compared to it, we can see that if a response without Chunked Encoding is received, then the payload will be handled by the ELSE block, where the payload_data_length will be correctly set up.

                        That is to say, WICED SDK doesn't support Chunked Encoding temporarily. And because the payload of a Chunked Encoding response won't be correctly parsed, the http_response won't carry the correct structure information. This explains why you always get 0 in payload_data_length.

                        To solve this, you can implement the TODO youself, trying to correctly parse the chunked response. You can do it in two ways:

                        1. Return each chunked response immediately as they are not chunked. This is the easier approach but it makes your response fragmentary.
                        2. Stitch all the chunked responses and then return in bulk. This makes the response to be expected but it requires complex handling methodology because the payload resides across multiple responses and sometimes the responses of a payload can be endless.

                        Best regards

                        2 of 2 people found this helpful
                        • 9. Re: WICED 101 httpbin_post_tls
                          MaCa_2922036

                          Do any patch or a new version of the HTTP_client lib is planned to be released ?

                          That's really a bad point to not have the chunked encoding supported in the library ...