WICED 101 httpbin_post_tls

Tip / Sign in to post questions, reply, level up, and achieve exciting badges. Know more

cross mob
RoMc_863046
Level 3
Level 3
25 sign-ins 10 replies posted 10 sign-ins

Hello,

I am able to compile and get a response from httpbin.org using the example in WICED 101 07b_06.  This is the project titled httpbin_post_tls.

I have modified the example to send a JSON sting to MY server instead of httpbin.  I have put the necessary certificates for my server in place.  I have verified with the server admin that my POST has been received and the server is sending the appropriate response, however, I am not receiving the payload in WICED.  I do receive the HTTP response header, just no payload (body) data.

I have this working with a different module manufacturer and I compared the HTTP response headers.  They are identical.

Is there an additional step I need to take in order to receive the payload?

The HTTP response indicates that it is chunked:

HTTP/1.1 200 OK

Server: nginx/1.14.0 (Ubuntu)

Date: Fri, 11 Sep 2020 14:36:56 GMT

Content-Type: application/json

Transfer-Encoding: chunked

Connection: keep-alive

Expires: Thu, 19 Nov 1981 08:52:00 GMT

Access-Control-Allow-Credentials: true

Access-Control-Allow-Origin:

Access-Control-Allow-Headers: X-Requested-With, Content-Type, Accept, Origin, Authorization

Access-Control-Allow-Methods: GET, POST, PUT, PATCH, DELETE, OPTIONS

Cache-Control: no-store, no-cache, must-revalidate, max-age=0, post-check=0, pre-check=0

Pragma: no-cache

Set-Cookie: PHPSESSID=5e2ojh23ud2fc09kn63hrrcdua; path=/

Strict-Transport-Security: max-age=31536000

Referrer-Policy: same-origin

X-Frame-Options: SAMEORIGIN

X-XSS-Protection: 1; mode=block

X-Content-Type-Options: nosniff

X-Frame-Options: SAMEORIGIN

Thank you,

-Rocko

0 Likes
1 Solution

Hi RoMc_863046

I finally reached to the conclusion that the issue resides in the WICED APIs. And I'm using WICED Studio 6.4.

Line 513 In the file <WICED_SDK>\43xxx_Wi-Fi\libraries\protocols\HTTP_client\http_client.c (i.e. the file that implement HTTP Client features) explains all. See below:

pastedImage_0.png

Since your server enabled Chunked Encoding, the payload of the response data should be handled in the IF block. But we can see there is no implementation in that block. Only a TODO is left. Compared to it, we can see that if a response without Chunked Encoding is received, then the payload will be handled by the ELSE block, where the payload_data_length will be correctly set up.

That is to say, WICED SDK doesn't support Chunked Encoding temporarily. And because the payload of a Chunked Encoding response won't be correctly parsed, the http_response won't carry the correct structure information. This explains why you always get 0 in payload_data_length.

To solve this, you can implement the TODO youself, trying to correctly parse the chunked response. You can do it in two ways:

  1. Return each chunked response immediately as they are not chunked. This is the easier approach but it makes your response fragmentary.
  2. Stitch all the chunked responses and then return in bulk. This makes the response to be expected but it requires complex handling methodology because the payload resides across multiple responses and sometimes the responses of a payload can be endless.

Best regards

View solution in original post

9 Replies
Charles_Lai
Moderator
Moderator
Moderator
500 replies posted 250 solutions authored 250 sign-ins

Hi,

I think this is the code example you are using.

CypressAcademy_WW101_Files/Projects/ww101key/07b/06_httpbin_post_tls at master · cypresssemiconducto...

To help with the debugging, please provide the test result that you run this example without modification on the same board, to see if it can get the expected payload from httpbin.org.

This step can help you analyze which side the problem exists in, the server or the client.

Best regards

0 Likes

Hello ChunleiL_51,

Here is the output from httbin_post_tls with httpbin.org payload.

Starting WICED vWiced_006.002.001.0002

Platform LAIRD_EWB initialised

Started ThreadX v5.8

Initialising NetX v5.10_sp3

Creating Packet pools

WLAN MAC Address : C0:EE:40:70:35:61

WLAN Firmware    : wl0: Apr 30 2018 04:14:19 version 7.45.98.50 (r688715 CY) FWID 01-283fcdb9

WLAN CLM         : API: 12.2 Data: 9.10.158 Compiler: 1.29.4 ClmImport: 1.36.3 Creation: 2019-05-07 05:05:14

Joining : bacon

Successfully joined : bacon

Obtaining IPv4 address via DHCP

DHCP CLIENT hostname WICED IP

IPv4 network ready IP: 192.168.1.155

Resolving IP address of httpbin.org

httpbin.org is at 3.221.81.55

Root Certificate Initialized

Connected to httpbin.org

------------------ Received response: 1 ------------------

----- Response Header: -----

HTTP/1.1 200 OK

Date: Mon, 14 Sep 2020 13:40:47 GMT

Content-Type: application/json

Content-Length: 371

Connection: keep-alive

Server: gunicorn/19.9.0

Access-Control-Allow-Origin: *

Access-Control-Allow-Credentials: true

----- Response Payload: -----

------------------ Received response: 1 ------------------

----- Response Payload: -----

{

  "args": {},

  "data": "{\"WICED\":\"yes\"}",

  "files": {},

  "form": {},

  "headers": {

    "Content-Length": "15",

    "Content-Type": "application/json",

    "Host": "httpbin.org",

    "X-Amzn-Trace-Id": "Root=1-5f5f72df-1c33ba8001dbf6da89028885"

  },

  "json": {

    "WICED": "yes"

  },

  "origin": "47.4.209.115",

  "url": "https://httpbin.org/post"

}

------------------ End Response 1 ------------------

-Rocko

0 Likes

Hi Rocko,

Just a hint, maybe it helps...

If you print the response->payload_data_length, it might indicate that it's 0.

However if you do for example a:

      print_content( (char*) response->payload, 10);      // 10 = magic number

...you'll notice the payload is present.

If this is indeed the case, you can consider creating a function that parses the header - similar to http_parse_header - which also looks for "Content-Length:" inside and returns the length of the body.

Please disregard my post if I'm completely off the track.

Good luck,

V

   Extracting the body length from the header:

                uint16_t value;

                    char *ptr = response->response_hdr;

                    while (ptr) {

                        ptr = strstr(ptr, "Content-Length: ");

                        if (ptr == NULL) {

                            break;

                        }

                        ptr+=sizeof("Content-Length: ")-1;

                        value = strtol(ptr, &ptr, 10);

                        printf("%lu\n", value);

                        continue;

                    }

RoMc_863046

Hi RoMc_863046

I finally reached to the conclusion that the issue resides in the WICED APIs. And I'm using WICED Studio 6.4.

Line 513 In the file <WICED_SDK>\43xxx_Wi-Fi\libraries\protocols\HTTP_client\http_client.c (i.e. the file that implement HTTP Client features) explains all. See below:

pastedImage_0.png

Since your server enabled Chunked Encoding, the payload of the response data should be handled in the IF block. But we can see there is no implementation in that block. Only a TODO is left. Compared to it, we can see that if a response without Chunked Encoding is received, then the payload will be handled by the ELSE block, where the payload_data_length will be correctly set up.

That is to say, WICED SDK doesn't support Chunked Encoding temporarily. And because the payload of a Chunked Encoding response won't be correctly parsed, the http_response won't carry the correct structure information. This explains why you always get 0 in payload_data_length.

To solve this, you can implement the TODO youself, trying to correctly parse the chunked response. You can do it in two ways:

  1. Return each chunked response immediately as they are not chunked. This is the easier approach but it makes your response fragmentary.
  2. Stitch all the chunked responses and then return in bulk. This makes the response to be expected but it requires complex handling methodology because the payload resides across multiple responses and sometimes the responses of a payload can be endless.

Best regards

Do any patch or a new version of the HTTP_client lib is planned to be released ?

That's really a bad point to not have the chunked encoding supported in the library ...

0 Likes

ChunleiL,

I initiated a MyCase to allow the sharing of Wireshark captures that may contain sensitive and competitive information.

Please reply to me outside of the Community if I can provide additional information or support.

When this issue is resolved, I trust we can summarize the resolution back here in this Community Post to help others who may experience a similar issue.

Greg C

0 Likes
RoMc_863046
Level 3
Level 3
25 sign-ins 10 replies posted 10 sign-ins

Thank you both for your feedback.

V,  You are correct.  When I print the body data with a "magic number" my payload IS there.  This is great news!

Unfortunately, The header does not have a "Content-Length" field.  My full response header is in the original post.

I am in the process of migrating from a non Cypress WiFi module.  When using the other module, I leveraged the nodejs http-parser library.  That parser was able to determine payload length without a "Content-Length" field.  I will have to determine a different method for finding the content length.  This field is not always a guarantee, especially as servers migrate to HTTP/2.

Thanks again.  If you have any ideas to help out with finding the payload length without the Content-Length field, other than bringing in the nodejs http-parser, please let me know.

-Rocko

VaSt_4778721

ChunleiL_51

0 Likes

Appreciate the help VaSt_4778721

0 Likes