HTTP daemon chunking of large files doesn't work.

Tip / Sign in to post questions, reply, level up, and achieve exciting badges. Know more

cross mob
Anonymous
Not applicable

With WICED SDK 3.1.2 and the BCM943362WCD4_3 development board I am unable to get the HTTP server example to be able to chunk large files over ~40kb. When debugging with wireshark I see the first chunk come through but then the next chunk immediately afterwards is the end chunk of "0" bytes.

Is there some bug with the HTTP server that it cannot support chunking larger resource files?

0 Likes
9 Replies
SeyhanA_31
Employee
Employee
250 replies posted 100 replies posted 50 replies posted

Hi,

Perhaps you can try the fix in this thread LWIP fail to transfer large files (HTTP requests)

Seyhan

0 Likes
Anonymous
Not applicable

That fix may work for the FreeRTOS user but I am running into this issue with NetX. I will try to use FreeRTOS with LWIP and see if I get different results.

0 Likes
Anonymous
Not applicable

I think I figured out the issue. In the http_server.c under wiced_result_t wiced_http_chunk_response the data_length_string for chunking can only be 5 characters long limiting the user to at most "FFFF" bytes of data. If you increase the size of this buffer you can get larger files through.

One Wireshark I was seeing D74C as the size of the chunk when in fact it should be 1D74C

0 Likes
Anonymous
Not applicable

This still had not been resolved as my fix to the affected function did not work. I am still seeing over wireshark that the chucking header from the wifi module to be D74C and not 1D74C which makes me think that there is something that is casting the uint32 into a uint16. I am compiling for NetX and using the latest version of the WICED SDK so the "Correct Answer" that was supply is not in fact the correct answer as it pertains to using FreeRTOS and LWIP.

I will investigate more but I would it if a Broadcom engineer can reproduce this issue. You just need to try and transfer a file over the HTTP daemon that is larger than 0xFFFF bytes.

0 Likes
Anonymous
Not applicable

Bug appears that it is in wiced_tcp_stream_write_resource

It is casting the size of the file to 16bits so this means that the chunking algorithm will never get it. Also means that no resource on the micro controller can be larger than 0xFFFF...

0 Likes
Anonymous
Not applicable

Another bug is in tcpip.c

in wiced_tcp_stream_write

/* Update variables */

        data_length                           = (uint16_t) (data_length - amount_to_write);

The data length should not be cast into a 16 bit integer after the first loop through. There is a good change that the data_length is more than 2*0xFFFF so doing this limits the size of a resource getting outputted though this function to 131070 bytes.

Simply remove the casting solves the problem.

/* Update variables */

        data_length                           = data_length - amount_to_write;

Still not sure if this will solve all my problems, but getting there at least the chunking call back function is getting the full data_length.

kgillespie wrote:

Another bug is in tcpip.c

in wiced_tcp_stream_write

/* Update variables */

        data_length                           = (uint16_t) (data_length - amount_to_write);

The data length should not be cast into a 16 bit integer after the first loop through. There is a good change that the data_length is more than 2*0xFFFF so doing this limits the size of a resource getting outputted though this function to 131070 bytes.

Simply remove the casting solves the problem.

/* Update variables */

        data_length                           = data_length - amount_to_write;

Still not sure if this will solve all my problems, but getting there at least the chunking call back function is getting the full data_length.

Hi Cypress team, mwf_mmfae​,

This is an old thread, but kgillespie's comment looks good to me.

I think the same issue still in SDK-3.7.0:

In wiced_tcp_stream_write:  (WICED/network/wiced_tcpip_common.c)

Current code has:

        /* Update variables */

        data_length                           = (uint16_t)(data_length - actual_amount_to_write);

        tcp_stream->tx_packet_space_available = (uint16_t) ( tcp_stream->tx_packet_space_available - actual_amount_to_write );

        data                                  = ((char*)data + actual_amount_to_write);^M

So why it need to cast to uint16_t before assign to data_length? // data_length is uint32_t

Any chance to take a look?

0 Likes
Anonymous
Not applicable

After patching all the issues I found I still can't get my entire file to get delivered. Not sure if I am hitting up against some other limitation. I guess I wasn't clear in my last post I am editing the file WICED/network/NetX/WICED/tcpip.c and was where I was finding most of the problems, mostly around needlessly recasting the data_length to 16bit. Though I am unsure what the rest of the problem is. I am now seeing that the chunking function is getting the complete size, but I am not getting all the information. It looks like wiced_tcp_stream_write is correctly dividing the data and sending it out.

0 Likes
Anonymous
Not applicable

I've been hit by this as well. Is it going to be fix in the next release?

0 Likes