Files of more than 200KB get corrupted when downloaded via https in WICED 5.2

[Anonymous]

I couldn't download a file larger than about 200KB using a secure WICED 5.2 connection, so I went back to your snip.https_client and created a simple program that replicated the problem on your BCM943907AEVAL1F board. The program downloads a 909,740 byte file and compares it to a previously saved file that is identical. This program successfully downloads every byte when built using WICED 5.1, but when using WICED 5.2, after about 200KB where the downloaded bytes exactly match the previously saved bytes, the downloaded bytes become different from the previously saved bytes. Using a terminal that displays 100 characters in the horizontal this is the console output:

WICED 5.1:

Starting WICED v5.1.0

Platform CYW943907AEVAL1F initialised

Started ThreadX v5.6

Initialising NetX_Duo v5.7_sp2

Creating Packet pools

WLAN MAC Address ****

WLAN Firmware    : wl0: Jul 12 2017 14:22:51 version 7.15.168.90 (r665791) FWID 01-772425

WLAN CLM         : API: 12.1 Data: 7.17.2 Compiler: 1.26.3 ClmImport: 1.26.12 Creation: 2015-01-10 11:06:18 Inc Data: 9.10.48 Inc Compiler: 1.31.3 Inc ClmImport: 1.36.3 Creation: 2017-07-12 14:21:10

Joining : ****

Successfully joined : ****

Obtaining IPv4 address via DHCP

DHCP CLIENT hostname WICED IP

IPv4 network ready IP: ****

Setting IPv6 link-local address

IPv6 network ready IP: ****

Resolving IP address of HTTPS server

Server is at ****

Getting '/'...

Read the certificate Key from DCT

About to Connect.

waiting for HTTP reply

One tick mark "|" is displayed for every packet downloaded.

||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||

||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||

||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||

||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||

||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||

||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||

||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||

||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||

||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||

||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||

||||||||||||||||||||||||||||

Server returned

WICED 5.2

Starting WICED vWiced_005.002.000.0022

Platform BCM943907AEVAL1F initialised

Started ThreadX v5.6

Initialising NetX_Duo v5.7_sp2

Creating Packet pools

WLAN MAC Address : ****

WLAN Firmware    : wl0: Jul 21 2017 03:05:13 version 7.15.168.92 (r666012) FWID 01-772425

WLAN CLM         : API: 12.2 Data: 9.10.49 Compiler: 1.31.3 ClmImport: 1.36.3 Creation: 2017-07-21 03:01:15

Joining : ***

Successfully joined : ****

Obtaining IPv4 address via DHCP

DHCP CLIENT hostname WICED IP

IPv4 network ready IP: ****

Setting IPv6 link-local address

IPv6 network ready IP: *****

Resolving IP address of HTTPS server

Server is at *****

Getting '/'...

Read the certificate Key from DCT

About to Connect.

waiting for HTTP reply

One tick mark "|" is displayed for every packet downloaded.

||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||

||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||

||||||||||||||||||

Corrupted data found starting at byte number 196240. Displayed as Valid byte = Downloaded byte

E0=24 28=2A 40=9D 07=0F 08=4C 28=D2 3A=69 07=34 08=13 28=C1 3C=14 07=58 08=23 28=3E 3F=E0 07=28

08=40 28=07 3D=08 07=28 08=3A 28=07 3B=08 07=28 08=3C 28=07 3E=08 07=28 08=3F 68=07 41=08 07=28

08=3D 4F=07 EA=08 73=28 00=3B 80=07 EA=08 02=28 01=3E 01=07 F0=08 AA=68 31=41 4A=07 40=08 48=4F

...

Get failed: 4

i have attached the code where I removed our URLs, filenames and security keys and replaced them with ***, so you will need to fix that before the code will run.

Build using "make https_client-BCM943907AEVAL1F download download_apps run"

NOTE: I copied wiced_https_get() from libraries/protocols/HTTP/http.c and then modified it so that it would download a 909,740 byte file and compare that to an identical copy that was saved in APP1 using download_apps when the project was built. I also used an older version of https_client, because the version that came with WICED 5.2 could not connect at all. Although, in hindsight it looks to me like the problem with WICED 5.2's https_client is probably nothing more that a bad hard coded certificate.

[AxLi_1746341]

Hi grsr​

You might want to add this one to your blog post for tracking wi-fi security issue.

[user_2177781]

Have you tried on a different ISP? It sounds like a the same, or at least a similar to one of the issue that I have been looking into.

wiced_tcp_stream_read issues in WICED 5.2

[Anonymous]

The program works fine for downloading files around half a MB in WICED 5.1, but it does not work in WICED 5.2. Consequently the problem has nothing to do with the file server. Before reproducing the issue on the BCM943907AEVAL1F I had the same problem with a custom board using an ISM43340 module with a more complicated custom OTA program. This custom board uses a different MCU and has a different RAM size. Since the problem always occurs at byte 196240 of the download I don't believe it has anything to do with a heap or stack overflow. I suspect there is some kind of counter that gets incremented as you increase the number of bytes downloaded, and eventually that counter has a numeric overflow. I have no idea where the numeric overflow is occurring and I could be completely wrong about it being a numeric overflow.

[AxLi_1746341]

webmstreric wrote:

The program works fine for downloading files around half a MB in WICED 5.1, but it does not work in WICED 5.2. 

Does it work with WICED-Studio 6.0?

[Anonymous · Accepted solution]

The program works fine in WICED 6.0.0. The problem appears to be fixed.

[AxLi_1746341]

webmstreric wrote:

The program works fine in WICED 6.0.0. The problem appears to be fixed.

Good to know that.

BTW, The SDK currently has below setting in include/wiced_defaults.h

#define WICED_TLS_MINOR_VERSION_MIN           (2)   /* Refers to TLS version 1.2. Values for TLS Versions: 0 ==> TLS v1.0, 1 ==> TLS v1.1, 2 ==> TLS v1.2 */

#define WICED_TLS_MINOR_VERSION_MAX           (2)   /* Refers to TLS version 1.2. Values for TLS Versions: 0 ==> TLS v1.0, 1 ==> TLS v1.1, 2 ==> TLS v1.2 */

I would suggest you change it to below for better testing coverage.

#define WICED_TLS_MINOR_VERSION_MIN           (0)   /* Refers to TLS version 1.2. Values for TLS Versions: 0 ==> TLS v1.0, 1 ==> TLS v1.1, 2 ==> TLS v1.2 */

#define WICED_TLS_MINOR_VERSION_MAX           (2)   /* Refers to TLS version 1.2. Values for TLS Versions: 0 ==> TLS v1.0, 1 ==> TLS v1.1, 2 ==> TLS v1.2 */