- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
I am testing snip.httpbin_org on CWY943907AEVAL1F. I use WICED-Studio-6.2.
The original sample work correctly. I modified the original source code to send data to my local server. I removed the CA check.
I can send small file (size = 300kbyte). but I could not send a file that is over 400kbyte size.
Is it needed to change mbedtls/config.h ?
Solved! Go to Solution.
- Labels:
-
WICED Studio
-
Wifi+Bluetooth
- Tags:
- tls client
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Please increase the packet pool size to avoid the assert issues.
GLOBAL_DEFINES += TX_PACKET_POOL_SIZE=xx \
RX_PACKET_POOL_SIZE=xx
The value of packet pools is dependent on the amount of data exchanged. Hence the value "xx" will be higher for applications where large amount of data is transferred.
+= TX_PACKET_POOL_SIZE=14 \
RX_PACKET_POOL_SIZE=12
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hello,
Root CA is to ensure the authenticity of the server. If you do not want a secure conneection, you can omit the certificates. In general, when a client wants to connect to a server, it sends a connection request. The server then sends a certificate in the server hello message. The client can verify the authenticity of the server as a trusted root ca is present with the client.
As you were already able to establish a connection with your local server, I believe you are familiar with the flow. Can you give more details about the failure of sending the 400kB file? At what point it fails? Is your connection still intact with the server? Do you specifically have problems with "POST" request or with "GET" request only?
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Thank you for your replay.
I use "400k.bin" that is filled with "0x35". The size is just 400KB. I use "GET" request.
I am checking the payload_data_length. The server finished sending the data. But Device received only 16384byte data.
When I checked by debug mode. "wiced_assert" was called in tls_host_create_buffer function.
I have checked "PUT" request . It's OK. But I have not checked "POST" request.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hello,
Can you please post the initial response from the server where the size of the file that needs to be transferred is passed?
I have set up a local server with some text to replicate the issue. I was able to receive the files up to 510KB. Please check the response I received form the server:
HTTP Header Information for response2 :
==============================================
HTTP/1.1 200 OK
server: ecstatic-3.3.0
cache-control: max-age=3600
last-modified: Wed, 21 Nov 2018 10:22:43 GMT
etag: W/"23925373020530490-512656-2018-11-21T10:22:43.372Z"
content-length: 512656
content-type: text/html; charset=UTF-8
Date: Wed, 21 Nov 2018 13:17:34 GMT
Connection: keep-alive
==============================================
Note: I am testing with httpbin_org snip with no security to check the variable size issue. I have not modified the variable size to uint32_t.
I can share the snip and local server details if required.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hello riya
> Can you please post the initial response from the server where the size of the file that needs to be transferred is passed?
I got a log with three patterns.
I changed httpbin_org.c slightly.
The value of response-> payload_data_length is added to another variable and it is printed at the end.
(This is the line "recv_total_size = ....".)
* httpbin_org.c is attached for reference.
File size is below.
1) 5MB
-> 1024 * 1024 * 5 = 5242880 = 0x00500000
3) 5MB + 1Byte
-> 1024 * 1024 * 5 + 1 = 5242881 = 0x00500001
2) 5MB - 1Byte
-> 1024 * 1024 * 5 - 1 = 5242879 = 0x004FFFFF
The logs is below.
1) 5MB
HTTP Header Information for response1 :
==============================================
HTTP/1.1 200 OK
Server: 04WebServer/1.92
Date: Thu, 22 Nov 2018 04:16:20 GMT
Content-Type: application/octet-stream
Connection: Close
Last-modified: Fri, 12 Oct 2018 05:10:16 GMT
ETag: "500000-5bc02cb8:5237"
Content-Length: 5242880
==============================================
Received total payload data for response1
recv_total_size = 0
2) 5MB + 1Byte
HTTP Header Information for response1 :
==============================================
HTTP/1.1 200 OK
Server: 04WebServer/1.92
Date: Thu, 22 Nov 2018 04:16:52 GMT
Content-Type: application/octet-stream
Connection: Close
Last-modified: Thu, 22 Nov 2018 04:06:40 GMT
ETag: "500001-5bf62b50:5237"
Content-Length: 5242881
==============================================
Received total payload data for response1
recv_total_size = 5242881
3) 5MB - 1Byte
HTTP Header Information for response1 :
==============================================
HTTP/1.1 200 OK
Server: 04WebServer/1.92
Date: Thu, 22 Nov 2018 04:17:26 GMT
Content-Type: application/octet-stream
Connection: Close
Last-modified: Thu, 22 Nov 2018 04:07:11 GMT
ETag: "4fffff-5bf62b6f:5237"
Content-Length: 5242879
==============================================
Received total payload data for response1
recv_total_size = 5242879
> I was able to receive the files up to 510KB. Please check the response I received form the server:
The response Content-Length is correct.
It is because it directly printed the character string in the header.
The actual length of the data received by the application is the sum of "response-> payload_data_length".
Also, I think that this problem will not be reproduced by the size of the test data of your log.
Since 512656(Byte) is 0x7D290(Byte), the values of the lower 16bits are not taken into account.
Please try with data whose lower 16bits of the file size will be the following numerical values.
0x0000: for example 5 MB (0x00500000)
0x4000: for example 400 KB (0x00064000)
Regards
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hello,
Local Server details:
Started a local server through cgywin using package http-server
1. Install the package:
npm install http-server -g
2. Create a dire named public and place the sometext_409600 file attached here. The sometext_409600.html file contains some junk data.
mkdir public
3. start http server
http-server ./public
I tested the httbin_org which you have attached and I got the following response:
Connecting to 192.168.43.121
Connected
HTTP Header Information for response1 :
==============================================
HTTP/1.1 200 OK
server: ecstatic-3.3.0
cache-control: max-age=3600
last-modified: Tue, 27 Nov 2018 13:02:55 GMT
etag: W/"3659174697948289-409600-2018-11-27T13:02:55.485Z"
content-length: 409600
content-type: text/html; charset=UTF-8
Date: Wed, 28 Nov 2018 05:34:17 GMT
Connection: keep-alive
==============================================
Received total payload data for response1
recv_total_size = 409600
Note: I have changed the SERVER_PORT to 8080.
Can you please attach the 5MB file that you are using for testing? I will test it on my side and try to replicate your issue.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hello
> Can you please attach the 5MB file that you are using for testing? I will test it on my side and try to replicate your issue.
I attached 5MB file.
Please change the file name so that it is easy to understand.
Thanks for the information on the details of the local server.
I have not prepared the server yet, but I tried it with IIS.
(Tested with Non-Secure)
--------------------------------------------------------------------------------------------
Connecting to 192.168.2.101
Connected
HTTP Header Information for response1 :
==============================================
HTTP/1.1 200 OK
Content-Type: text/html
Last-Modified: Fri, 12 Oct 2018 05:10:16 GMT
Accept-Ranges: bytes
ETag: "efa66dde961d41:0"
Server: Microsoft-IIS/7.5
Date: Thu, 29 Nov 2018 06:17:05 GMT
Content-Length: 5242880
==============================================
Received total payload data for response1
recv_total_size : 0
Disconnected from 192.168.2.101
--------------------------------------------------------------------------------------------
You tested it without using the patch from https_client.c I attached before, right?
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hello
I tried some additional tests.
I've sorted out my test results so far.
Secure or Non-Secure | FileSize | Result OK : Issue Not Occured NG : Issue Occred, |
---|---|---|
Non-Secure | 5MB | NG |
Non-Secure | 5MB - 1Byte | OK |
Non-Secure | 5MB + 1Byte | OK |
Non-Secure | 400KB | OK |
Secure | 5MB | NG |
Secure | 400KB | NG |
On my side, with 5MB file, issue is occured both Secure and Non-Secure.
With 400KB file, no problem occurred when Non-Secure, but a problem occurred when it was Secure.
・Non-Secure-400KB Response Log
--------------------------------------------------------------
HTTP Header Information for response1 :
==============================================
HTTP/1.1 200 OK
Server: 04WebServer/1.92
Date: Thu, 29 Nov 2018 06:38:34 GMT
Content-Type: application/octet-stream
Connection: Close
Last-modified: Mon, 12 Nov 2018 04:28:48 GMT
ETag: "64000-5be90180:5237"
Content-Length: 409600
==============================================
Received total payload data for response1
recv_total_size : 409600
Disconnected from 192.168.2.101
--------------------------------------------------------------
・Secure-400KB Response Log
--------------------------------------------------------------
HTTP Header Information for response1 :
==============================================
HTTP/1.1 200 OK
Server: 04WebServer/1.92
Date: Thu, 29 Nov 2018 06:40:09 GMT
Content-Type: application/octet-stream
Connection: Close
Last-modified: Mon, 12 Nov 2018 04:28:48 GMT
ETag: "64000-5be90180:5237"
Content-Length: 409600
==============================================
Received total payload data for response1
recv_total_size : 16384
--------------------------------------------------------------
Regards
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
I think we figured out the problem in receiving large file i.e., 5MB(NS).
The http_library has a function for capturing the content length from the http response. The value is type casted to uint16_t which should be changed to uint32_t.
Kindly change the data type at the following places:
1. In file http_client.c, change static uint16_t get_content_length( char* data , uint16_t data_len) to static uint32_t get_content_length( char* data , uint16_t data_len)
2. change return ( (uint16_t) ( strtol( data + 1, NULL, 10 ) ) ); to return ( (uint32_t) ( strtol( data + 1, NULL, 10 ) ) );
3. Change the content length variable in static wiced_result_t client_receive_handler( void* arg 😞
uint32_t content_length = 0;
4. In file http_client.h change the payload_data_length in structure http_response_t
uint32_t payload_data_length; /* This length indicates only payload length */ //uint16_t
Please check with these changes and revert back the result.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
riya wrote:
I think we figured out the problem in receiving large file i.e., 5MB(NS).
The http_library has a function for capturing the content length from the http response. The value is type casted to uint16_t which should be changed to uint32_t.
Isn't this the same finding nunokawa.akira_2072836 already told you on Oct 15, 2018 in this thread?
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Yes Sir but there was still problem in sending 5MB data(5242880 bytes) which goes through if the mentioned changes are done.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
This fix is for http_client.c that has not changed anything, right?
If so, for example to change (1) and (2), it seems necessary to change other variables to 32 bit.
For example, response_info-> total_remaining_length or http_response.remaining_length.
Please provide a patch file as I did, not a procedure on how to fix it.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Besides, where does the connection fail in secure mode. Please capture the TLS handshake information as mentioned in Debugging TLS in WICED and share the log file.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Do you have anything to do with the modification of http_client.c and TLS?
I think TLS is related to the problem where wiced_assert occurs in wiced_tls.c.
If so, look at that log as it already shares it to you in another way.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Please increase the packet pool size to avoid the assert issues.
GLOBAL_DEFINES += TX_PACKET_POOL_SIZE=xx \
RX_PACKET_POOL_SIZE=xx
The value of packet pools is dependent on the amount of data exchanged. Hence the value "xx" will be higher for applications where large amount of data is transferred.
+= TX_PACKET_POOL_SIZE=14 \
RX_PACKET_POOL_SIZE=12
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
PriyaM_16 wrote:
Please increase the packet pool size to avoid the assert issues.
GLOBAL_DEFINES += TX_PACKET_POOL_SIZE=xx \
RX_PACKET_POOL_SIZE=xx
The value of packet pools is dependent on the amount of data exchanged. Hence the value "xx" will be higher for applications where large amount of data is transferred.
+= TX_PACKET_POOL_SIZE=14 \
RX_PACKET_POOL_SIZE=12
Generally you won't know exactly the amount of data exchanged by every user. You cannot avoid the assert issues this way.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
I made simple files. I have checked on AWS S3.
HTTPS Get
File Name "5M.bin" the file size is just 5MByte , filled with 0xFF.
-> NG
File Name "5M1.bin" the file size is 5MByte +1 Byte , filled with 0xFF
-> OK
I don't call "wiced_tls_init_root_ca_certificates" function in "application_start" function.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
It seems that the remaining data size is not managed properly.
Since the variable is uint16, it seems that it is doing abnormal operation by judging with only the lower 16 bits of the actual file size.
(For example,
5MB is 0x00500000, it seems that it is recognized as 0 byte on the program.
400KB is 0x00064000, so it seems that it is recognized as 16384 bytes on the program. )
Attach the modified file for reference.
43xxx_Wi-Fi\libraries\protocols\HTTP_client\http_client.c
This seems like a cause, but what about it?
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
I have tried the patch(http_client.c.zip). and It worked correctly.
But I checked between local server. It does not work correctly.
I run "testing snip.httpbin_org-debug download" by WICED debug.
the system stopped by "wiced_assert" at tls_host_create_buffer fucntion of wiced_tls.c.
And MBEDTLS_SSL_DEBUG_MSG( 1, ( "unknown record type" ) ) was called in ssl_tls.c.
Is this another issue?