1 2 Previous Next 16 Replies Latest reply on Dec 12, 2018 12:23 AM by nunokawa.akira_2072836

    https get can not work correctly

    test963local_3774766

      I am testing snip.httpbin_org on CWY943907AEVAL1F. I use WICED-Studio-6.2.

       

      The original sample work correctly. I modified the  original source code to send data to my local server. I removed the CA check.

      I can send small file (size = 300kbyte). but  I could not send a file that is over 400kbyte size.

       

      Is it needed to change mbedtls/config.h ?

        • 1. Re: https get can not work correctly
          riya

          Hello,

          Root CA is to ensure the authenticity of the server. If you do not want a secure conneection, you can omit the certificates. In general, when a client wants to connect to a server, it sends a connection request. The server then sends a certificate in the server hello message. The client can verify the authenticity of the server as a trusted root ca is present with the client.

          As you were already able to establish a connection with your local server, I believe you are familiar with the flow. Can you give more details about the failure of sending the 400kB file? At what point it fails? Is your connection still intact with the server? Do you specifically have problems with "POST" request or with "GET" request only?

          2 of 2 people found this helpful
          • 2. Re: https get can not work correctly
            test963local_3774766

            Thank you for your replay.

            I use "400k.bin" that is filled with "0x35".  The size is just 400KB. I use "GET" request.

            I am checking the payload_data_length.  The server finished sending the data. But Device received only 16384byte data.

             

            When I checked by debug mode. "wiced_assert"  was called in tls_host_create_buffer function.

            I have checked "PUT" request . It's OK. But I have not checked "POST" request.

            • 3. Re: https get can not work correctly
              test963local_3774766

              I made simple files.  I have checked on AWS S3.

               

              HTTPS Get

              File Name "5M.bin"  the file size is just 5MByte ,  filled with 0xFF.

              ->  NG 

               

              File Name "5M1.bin"  the file size is 5MByte +1 Byte , filled with 0xFF

              -> OK

               

              I don't call  "wiced_tls_init_root_ca_certificates" function in "application_start" function.

              • 4. Re: https get can not work correctly
                nunokawa.akira_2072836

                It seems that the remaining data size is not managed properly.

                Since the variable is uint16, it seems that it is doing abnormal operation by judging with only the lower 16 bits of the actual file size.

                (For example,

                5MB is 0x00500000, it seems that it is recognized as 0 byte on the program.

                400KB is 0x00064000, so it seems that it is recognized as 16384 bytes on the program. )

                 

                Attach the modified file for reference.

                43xxx_Wi-Fi\libraries\protocols\HTTP_client\http_client.c

                 

                This seems like a cause, but what about it?

                2 of 2 people found this helpful
                • 5. Re: https get can not work correctly
                  test963local_3774766

                  I have tried the patch(http_client.c.zip).  and It worked correctly.

                  But I checked between local server.  It does not work correctly.  

                   

                  I run "testing snip.httpbin_org-debug download"  by WICED debug.

                  the system stopped by "wiced_assert"  at tls_host_create_buffer fucntion of wiced_tls.c.

                  And MBEDTLS_SSL_DEBUG_MSG( 1, ( "unknown record type" ) )  was called in ssl_tls.c.

                   

                  Is this another issue?

                  • 6. Re: https get can not work correctly
                    riya

                    Hello,

                    Can you please post the initial response from the server where the size of the file that needs to be transferred is passed?

                    I have set up a local server with some text to replicate the issue. I was able to receive the files up to 510KB. Please check the response I received form the server:

                    HTTP Header Information for response2 :

                    ==============================================

                    HTTP/1.1 200 OK

                    server: ecstatic-3.3.0

                    cache-control: max-age=3600

                    last-modified: Wed, 21 Nov 2018 10:22:43 GMT

                    etag: W/"23925373020530490-512656-2018-11-21T10:22:43.372Z"

                    content-length: 512656

                    content-type: text/html; charset=UTF-8

                    Date: Wed, 21 Nov 2018 13:17:34 GMT

                    Connection: keep-alive

                    ==============================================

                     

                    Note: I am testing with httpbin_org snip with no security to check the variable size issue. I have not modified the variable size to uint32_t.

                    I can share the snip and local server details if required.

                    1 of 1 people found this helpful
                    • 7. Re: https get can not work correctly
                      nunokawa.akira_2072836

                      Hello riya

                       

                      > Can you please post the initial response from the server where the size of the file that needs to be transferred is passed?

                      I got a log with three patterns.

                       

                      I changed httpbin_org.c slightly.

                      The value of response-> payload_data_length is added to another variable and it is printed at the end.

                      (This is the line "recv_total_size = ....".)

                      * httpbin_org.c is attached for reference.

                       

                      File size is below.

                      1) 5MB

                        -> 1024 * 1024 * 5    = 5242880 = 0x00500000

                      3) 5MB + 1Byte

                        -> 1024 * 1024 * 5 + 1 = 5242881 = 0x00500001

                      2) 5MB - 1Byte

                        -> 1024 * 1024 * 5 - 1 = 5242879 = 0x004FFFFF

                       

                      The logs is below.

                      1) 5MB

                      HTTP Header Information for response1 :

                      ==============================================

                      HTTP/1.1 200 OK

                      Server: 04WebServer/1.92

                      Date: Thu, 22 Nov 2018 04:16:20 GMT

                      Content-Type: application/octet-stream

                      Connection: Close

                      Last-modified: Fri, 12 Oct 2018 05:10:16 GMT

                      ETag: "500000-5bc02cb8:5237"

                      Content-Length: 5242880

                      ==============================================

                      Received total payload data for response1

                      recv_total_size = 0

                       

                      2) 5MB + 1Byte

                      HTTP Header Information for response1 :

                      ==============================================

                      HTTP/1.1 200 OK

                      Server: 04WebServer/1.92

                      Date: Thu, 22 Nov 2018 04:16:52 GMT

                      Content-Type: application/octet-stream

                      Connection: Close

                      Last-modified: Thu, 22 Nov 2018 04:06:40 GMT

                      ETag: "500001-5bf62b50:5237"

                      Content-Length: 5242881

                      ==============================================

                      Received total payload data for response1

                      recv_total_size = 5242881

                       

                      3) 5MB - 1Byte

                      HTTP Header Information for response1 :

                      ==============================================

                      HTTP/1.1 200 OK

                      Server: 04WebServer/1.92

                      Date: Thu, 22 Nov 2018 04:17:26 GMT

                      Content-Type: application/octet-stream

                      Connection: Close

                      Last-modified: Thu, 22 Nov 2018 04:07:11 GMT

                      ETag: "4fffff-5bf62b6f:5237"

                      Content-Length: 5242879

                      ==============================================

                      Received total payload data for response1

                      recv_total_size = 5242879

                       

                      > I was able to receive the files up to 510KB. Please check the response I received form the server:

                      The response Content-Length is correct.
                      It is because it directly printed the character string in the header.

                      The actual length of the data received by the application is the sum of "response-> payload_data_length".

                       

                      Also, I think that this problem will not be reproduced by the size of the test data of your log.
                      Since 512656(Byte) is 0x7D290(Byte), the values of the lower 16bits are not taken into account.

                      Please try with data whose lower 16bits of the file size will be the following numerical values.
                      0x0000: for example 5 MB (0x00500000)
                      0x4000: for example 400 KB (0x00064000)

                       

                      Regards

                      • 8. Re: https get can not work correctly
                        riya

                        Hello,

                        Local Server details:

                        Started a local server through cgywin using package http-server

                        1. Install the package:

                            npm install http-server -g

                        2.  Create a dire named public and place the sometext_409600 file attached here. The sometext_409600.html file contains some junk data.

                            mkdir public

                        3. start http server

                            http-server ./public

                         

                        I tested the httbin_org which you have attached and I got the following response:

                         

                        Connecting to 192.168.43.121

                        Connected

                        HTTP Header Information for response1 :

                        ==============================================

                        HTTP/1.1 200 OK

                        server: ecstatic-3.3.0

                        cache-control: max-age=3600

                        last-modified: Tue, 27 Nov 2018 13:02:55 GMT

                        etag: W/"3659174697948289-409600-2018-11-27T13:02:55.485Z"

                        content-length: 409600

                        content-type: text/html; charset=UTF-8

                        Date: Wed, 28 Nov 2018 05:34:17 GMT

                        Connection: keep-alive

                        ==============================================

                        Received total payload data for response1

                        recv_total_size = 409600

                         

                        Note: I have changed the SERVER_PORT to 8080.

                         

                        Can you please attach the 5MB file that you are using for testing? I will test it on my side and try to replicate your issue.

                        1 of 1 people found this helpful
                        • 9. Re: https get can not work correctly
                          nunokawa.akira_2072836

                          Hello

                           

                          > Can you please attach the 5MB file that you are using for testing? I will test it on my side and try to replicate your issue.

                           

                          I attached 5MB file.

                          Please change the file name so that it is easy to understand.

                           

                          Thanks for the information on the details of the local server.

                          I have not prepared the server yet, but I tried it with IIS.

                          (Tested with Non-Secure)

                           

                          --------------------------------------------------------------------------------------------

                          Connecting to 192.168.2.101

                          Connected

                          HTTP Header Information for response1 :

                          ==============================================

                          HTTP/1.1 200 OK

                          Content-Type: text/html

                          Last-Modified: Fri, 12 Oct 2018 05:10:16 GMT

                          Accept-Ranges: bytes

                          ETag: "efa66dde961d41:0"

                          Server: Microsoft-IIS/7.5

                          Date: Thu, 29 Nov 2018 06:17:05 GMT

                          Content-Length: 5242880

                          ==============================================

                          Received total payload data for response1

                          recv_total_size : 0

                          Disconnected from 192.168.2.101

                          --------------------------------------------------------------------------------------------

                           

                          You tested it without using the patch from https_client.c I attached before, right?

                          • 10. Re: https get can not work correctly
                            nunokawa.akira_2072836

                            Hello

                             

                            I tried some additional tests.

                            I've sorted out my test results so far.

                             

                            Secure

                            or

                            Non-Secure

                            FileSize

                            Result

                              OK : Issue Not Occured

                              NG : Issue Occred,

                            Non-Secure5MBNG
                            Non-Secure5MB - 1ByteOK
                            Non-Secure5MB + 1ByteOK
                            Non-Secure400KBOK
                            Secure5MBNG
                            Secure400KBNG

                             

                            On my side, with 5MB file, issue is occured both Secure and Non-Secure.

                            With 400KB file, no problem occurred when Non-Secure, but a problem occurred when it was Secure.

                             

                             

                            ・Non-Secure-400KB Response Log

                            --------------------------------------------------------------

                            HTTP Header Information for response1 :

                            ==============================================

                            HTTP/1.1 200 OK

                            Server: 04WebServer/1.92

                            Date: Thu, 29 Nov 2018 06:38:34 GMT

                            Content-Type: application/octet-stream

                            Connection: Close

                            Last-modified: Mon, 12 Nov 2018 04:28:48 GMT

                            ETag: "64000-5be90180:5237"

                            Content-Length: 409600

                            ==============================================

                            Received total payload data for response1

                            recv_total_size : 409600

                            Disconnected from 192.168.2.101

                            --------------------------------------------------------------

                             

                             

                            ・Secure-400KB Response Log

                            --------------------------------------------------------------

                            HTTP Header Information for response1 :

                            ==============================================

                            HTTP/1.1 200 OK

                            Server: 04WebServer/1.92

                            Date: Thu, 29 Nov 2018 06:40:09 GMT

                            Content-Type: application/octet-stream

                            Connection: Close

                            Last-modified: Mon, 12 Nov 2018 04:28:48 GMT

                            ETag: "64000-5be90180:5237"

                            Content-Length: 409600

                            ==============================================

                            Received total payload data for response1

                            recv_total_size : 16384

                            --------------------------------------------------------------

                             

                            Regards

                            • 11. Re: https get can not work correctly
                              riya

                              I think we figured out the problem in receiving large file i.e., 5MB(NS).

                              The http_library has a function for capturing the content length from the http response. The value is type casted to uint16_t which should be changed to uint32_t.

                              Kindly change the data type at the following places:

                              1. In file http_client.c, change static uint16_t get_content_length( char* data , uint16_t data_len) to static uint32_t get_content_length( char* data , uint16_t data_len)

                              2. change return ( (uint16_t) ( strtol( data + 1, NULL, 10 ) ) ); to return ( (uint32_t) ( strtol( data + 1, NULL, 10 ) ) );

                              3. Change the content length variable in static wiced_result_t client_receive_handler( void* arg ):

                                uint32_t content_length = 0;

                              4. In file http_client.h change the payload_data_length in structure http_response_t

                                uint32_t payload_data_length; /* This length indicates only payload length */ //uint16_t

                               

                              Please check with these changes and revert back the result.

                              1 of 1 people found this helpful
                              • 12. Re: https get can not work correctly
                                riya

                                Besides, where does the connection fail in secure mode. Please capture the TLS handshake information as mentioned in Debugging TLS in WICED and share the log file.

                                • 13. Re: https get can not work correctly
                                  axel.lin_1746341

                                  riya wrote:

                                   

                                  I think we figured out the problem in receiving large file i.e., 5MB(NS).

                                  The http_library has a function for capturing the content length from the http response. The value is type casted to uint16_t which should be changed to uint32_t.

                                  Isn't this the same finding nunokawa.akira_2072836 already told you on Oct 15, 2018 in this thread?

                                  • 14. Re: https get can not work correctly
                                    riya

                                    Yes Sir but there was still problem in sending 5MB data(5242880 bytes) which goes through if the mentioned changes are done.

                                    1 2 Previous Next