- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hello,
I am able to compile and get a response from httpbin.org using the example in WICED 101 07b_06. This is the project titled httpbin_post_tls.
I have modified the example to send a JSON sting to MY server instead of httpbin. I have put the necessary certificates for my server in place. I have verified with the server admin that my POST has been received and the server is sending the appropriate response, however, I am not receiving the payload in WICED. I do receive the HTTP response header, just no payload (body) data.
I have this working with a different module manufacturer and I compared the HTTP response headers. They are identical.
Is there an additional step I need to take in order to receive the payload?
The HTTP response indicates that it is chunked:
HTTP/1.1 200 OK
Server: nginx/1.14.0 (Ubuntu)
Date: Fri, 11 Sep 2020 14:36:56 GMT
Content-Type: application/json
Transfer-Encoding: chunked
Connection: keep-alive
Expires: Thu, 19 Nov 1981 08:52:00 GMT
Access-Control-Allow-Credentials: true
Access-Control-Allow-Origin:
Access-Control-Allow-Headers: X-Requested-With, Content-Type, Accept, Origin, Authorization
Access-Control-Allow-Methods: GET, POST, PUT, PATCH, DELETE, OPTIONS
Cache-Control: no-store, no-cache, must-revalidate, max-age=0, post-check=0, pre-check=0
Pragma: no-cache
Set-Cookie: PHPSESSID=5e2ojh23ud2fc09kn63hrrcdua; path=/
Strict-Transport-Security: max-age=31536000
Referrer-Policy: same-origin
X-Frame-Options: SAMEORIGIN
X-XSS-Protection: 1; mode=block
X-Content-Type-Options: nosniff
X-Frame-Options: SAMEORIGIN
Thank you,
-Rocko
Solved! Go to Solution.
- Labels:
-
WICED Studio
-
Wifi+Bluetooth
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hi RoMc_863046
I finally reached to the conclusion that the issue resides in the WICED APIs. And I'm using WICED Studio 6.4.
Line 513 In the file <WICED_SDK>\43xxx_Wi-Fi\libraries\protocols\HTTP_client\http_client.c (i.e. the file that implement HTTP Client features) explains all. See below:
Since your server enabled Chunked Encoding, the payload of the response data should be handled in the IF block. But we can see there is no implementation in that block. Only a TODO is left. Compared to it, we can see that if a response without Chunked Encoding is received, then the payload will be handled by the ELSE block, where the payload_data_length will be correctly set up.
That is to say, WICED SDK doesn't support Chunked Encoding temporarily. And because the payload of a Chunked Encoding response won't be correctly parsed, the http_response won't carry the correct structure information. This explains why you always get 0 in payload_data_length.
To solve this, you can implement the TODO youself, trying to correctly parse the chunked response. You can do it in two ways:
- Return each chunked response immediately as they are not chunked. This is the easier approach but it makes your response fragmentary.
- Stitch all the chunked responses and then return in bulk. This makes the response to be expected but it requires complex handling methodology because the payload resides across multiple responses and sometimes the responses of a payload can be endless.
Best regards
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hi,
I think this is the code example you are using.
To help with the debugging, please provide the test result that you run this example without modification on the same board, to see if it can get the expected payload from httpbin.org.
This step can help you analyze which side the problem exists in, the server or the client.
Best regards
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hello ChunleiL_51,
Here is the output from httbin_post_tls with httpbin.org payload.
Starting WICED vWiced_006.002.001.0002
Platform LAIRD_EWB initialised
Started ThreadX v5.8
Initialising NetX v5.10_sp3
Creating Packet pools
WLAN MAC Address : C0:EE:40:70:35:61
WLAN Firmware : wl0: Apr 30 2018 04:14:19 version 7.45.98.50 (r688715 CY) FWID 01-283fcdb9
WLAN CLM : API: 12.2 Data: 9.10.158 Compiler: 1.29.4 ClmImport: 1.36.3 Creation: 2019-05-07 05:05:14
Joining : bacon
Successfully joined : bacon
Obtaining IPv4 address via DHCP
DHCP CLIENT hostname WICED IP
IPv4 network ready IP: 192.168.1.155
Resolving IP address of httpbin.org
httpbin.org is at 3.221.81.55
Root Certificate Initialized
Connected to httpbin.org
------------------ Received response: 1 ------------------
----- Response Header: -----
HTTP/1.1 200 OK
Date: Mon, 14 Sep 2020 13:40:47 GMT
Content-Type: application/json
Content-Length: 371
Connection: keep-alive
Server: gunicorn/19.9.0
Access-Control-Allow-Origin: *
Access-Control-Allow-Credentials: true
----- Response Payload: -----
------------------ Received response: 1 ------------------
----- Response Payload: -----
{
"args": {},
"data": "{\"WICED\":\"yes\"}",
"files": {},
"form": {},
"headers": {
"Content-Length": "15",
"Content-Type": "application/json",
"Host": "httpbin.org",
"X-Amzn-Trace-Id": "Root=1-5f5f72df-1c33ba8001dbf6da89028885"
},
"json": {
"WICED": "yes"
},
"origin": "47.4.209.115",
"url": "https://httpbin.org/post"
}
------------------ End Response 1 ------------------
-Rocko
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hi Rocko,
Just a hint, maybe it helps...
If you print the response->payload_data_length, it might indicate that it's 0.
However if you do for example a:
print_content( (char*) response->payload, 10); // 10 = magic number
...you'll notice the payload is present.
If this is indeed the case, you can consider creating a function that parses the header - similar to http_parse_header - which also looks for "Content-Length:" inside and returns the length of the body.
Please disregard my post if I'm completely off the track.
Good luck,
V
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Extracting the body length from the header:
uint16_t value;
char *ptr = response->response_hdr;
while (ptr) {
ptr = strstr(ptr, "Content-Length: ");
if (ptr == NULL) {
break;
}
ptr+=sizeof("Content-Length: ")-1;
value = strtol(ptr, &ptr, 10);
printf("%lu\n", value);
continue;
}
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hi RoMc_863046
I finally reached to the conclusion that the issue resides in the WICED APIs. And I'm using WICED Studio 6.4.
Line 513 In the file <WICED_SDK>\43xxx_Wi-Fi\libraries\protocols\HTTP_client\http_client.c (i.e. the file that implement HTTP Client features) explains all. See below:
Since your server enabled Chunked Encoding, the payload of the response data should be handled in the IF block. But we can see there is no implementation in that block. Only a TODO is left. Compared to it, we can see that if a response without Chunked Encoding is received, then the payload will be handled by the ELSE block, where the payload_data_length will be correctly set up.
That is to say, WICED SDK doesn't support Chunked Encoding temporarily. And because the payload of a Chunked Encoding response won't be correctly parsed, the http_response won't carry the correct structure information. This explains why you always get 0 in payload_data_length.
To solve this, you can implement the TODO youself, trying to correctly parse the chunked response. You can do it in two ways:
- Return each chunked response immediately as they are not chunked. This is the easier approach but it makes your response fragmentary.
- Stitch all the chunked responses and then return in bulk. This makes the response to be expected but it requires complex handling methodology because the payload resides across multiple responses and sometimes the responses of a payload can be endless.
Best regards
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Do any patch or a new version of the HTTP_client lib is planned to be released ?
That's really a bad point to not have the chunked encoding supported in the library ...
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
ChunleiL,
I initiated a MyCase to allow the sharing of Wireshark captures that may contain sensitive and competitive information.
Please reply to me outside of the Community if I can provide additional information or support.
When this issue is resolved, I trust we can summarize the resolution back here in this Community Post to help others who may experience a similar issue.
Greg C
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Thank you both for your feedback.
V, You are correct. When I print the body data with a "magic number" my payload IS there. This is great news!
Unfortunately, The header does not have a "Content-Length" field. My full response header is in the original post.
I am in the process of migrating from a non Cypress WiFi module. When using the other module, I leveraged the nodejs http-parser library. That parser was able to determine payload length without a "Content-Length" field. I will have to determine a different method for finding the content length. This field is not always a guarantee, especially as servers migrate to HTTP/2.
Thanks again. If you have any ideas to help out with finding the payload length without the Content-Length field, other than bringing in the nodejs http-parser, please let me know.
-Rocko
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content