- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Add while (1) { } around the wiced_https_get() call, then the device will
run to out-of-memory in a few minutes. then wiced_https_get() always fails.
Any chance Cypress AE team can take a look? I think this is an important issue.
I'm not sure if the memory leak is in TLS library or other part.
Axel
- Labels:
-
SDK 3.x
- Tags:
- memory leak
- tls
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
mwf_mmfae wrote:
I spoke to the development team and we should be able to release SDK 3.7.0-7 next week prior to Thanksgiving. This rev will have all of the patches which were applied in WICED Studio 4.
Hi mwf_mmfae
The SDK bug is a gating issue for us and all our resources depend on the timing to get SDK fix.
Our h/w and application features are ready, the only missing bit is the SDK bug that we cannot fix it by ourselves.
And we need to rearrange full test once we got new SDK.
It is not acceptable that the SDK release delay again and again.
I have to ask again. When will SDK 3.7.0-7 available?
Note, I don't ask any customization for us. I'm asking for SDK bug fixes only.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
I will continue to ask the development team for a time commitment on this release of the SDK.
I have asked twice today.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
mwf_mmfae wrote:
I will continue to ask the development team for a time commitment on this release of the SDK.
I have asked twice today.
Thanks, I hope to get sdk release soon.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hi mwf_mmfae,
What about the issue reported yesterday in the below mentioned discussion chain. Even I spend whole in testing my application on 4.0.1 hoping that you might have fixed the BESL issue but its so sad to see that we are still getting out_of_memory error after some no. of runs
Its annoying to see that after these much discussions also your team is not able to provide a good stable release for BESL Library without any memory leaks. Its something basic that a library you provide should not have any kind of memory leaks. Other bugs still its acceptable but not Memory leak
https://community.cypress.com/message/29721?et=watches.email.thread#29721
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
I think something very wrong in WICED SDK release process!
If cypress team can release fixes only for the SDK bugs (including source code fix or library fix) on the forum by patch or by zip files,
People don't need to wait for new sdk release. People can continue use the SDK they are working on.
Otherwise cypress team needs to frequently update sdk release.
However, current situation is: No bug fix in any form for very long time.
I use the SDK so I do care about the quality of the SDK.
I have tried asking to improve the process, but I don't see too much improvement.
e.g.
Let the core team review user provided patches on the forum.
For the person familiar with the code, it's easy to tell if a fix is correct or not.
Follow up the pending issues. As I have pointed out many issues are still pending without follow up actions.
No response from cypress, it's so silence.
And most importantly, cypress needs to provide bug fix for the binary library in older SDK.
Evey new sdk can become old sdk some doay. You can not keep asking users to upgrade SDK.
For example, all older 3.x SDK has problem connecting IIS server using TLS-v1.2 due to a bug in BESL library.
This means all wiced devices on the market using https has this problem.
I just cannot believe there is no fix from cypress for such issue.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hi mwf_mmfae,
If providing a memory leak-free TLS library is a real burden for your core team please think about alternative way to help customers --
providing alternative open-source TLS library adapter library as an option.
ex. mbed TLS (https://tls.mbed.org/ )
That customers can have another way to save their projects.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
According to the SW Team:
1.) Fixes in 4.0.1 release
- a. MQTT reconnect logic fixed up to allow application to initiate a reconnection whenever the connection goes down. Of course, the app has to handle the reconnection but the library reliably sends an event on disconnection.
- b. BESL memory leaks – There were 2 leaks fixed relating to BESL and these 2 fixes should have been present since 4.0 release
2.) Fixes in 3.7.0-7 release
- a. BESL memory leaks – Both the BESL leaks have been fixed.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
mwf_mmfae wrote:
2.) Fixes in 3.7.0-7 release
- a. BESL memory leaks – Both the BESL leaks have been fixed.
What about the BESL fix for sdk-3.1.2?
More and more of my customers are complaining the issue about not able to connect azure-iot (due to BESL bug).
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
mifo Tested latest-released WICED 4.1. Out of the box, the heap leak still occurs when using MQTT.
Despite closing the connection, my heap search algorithm still finds and frees quite a few instances of "bignum" "x509" and "tls".
Any thoughts?
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
HI Cypress Team,
Below you can find listing of calls for MQTT open and MQTT close routines (2 full cycles MQTT open-close) and info about mem allocation (malloc_info_command).
As you can see some MQTT close routine do not release all allocated memory by some MQTT open routine (probably mqtt_network_connect).
wiced_result_t mqtt_network_connect( mqtt_socket_t *socket )
{
return wiced_tcp_connect( &socket->socket, &socket->server_ip_address, socket->portnumber, WICED_MQTT_CONNECTION_TIMEOUT );
}
==> allocates about 3kB of RAM but wiced_tcp_disconnect do not release this allocated memory, so each open-close cycle "earns" 3kB up to "out memory" event or hang-on system.
It is very easy to find and check, so why up to now you have not a solution of problem or some workaround ?
Robert
ARENA | ALLOCATED | NON-INUSE | TOP-MOST | dARENA | dALLOC | LEAK | |
wiced_hostname_lookup | |||||||
###### mqtt_network_init / mqtt_network_init | 27756 | 26900 | 856 | 856 | |||
###### mqtt_network_init / wiced_tcp_create_socket | 27756 | 26900 | 856 | 856 | 0 | 0 | |
###### mqtt_network_init / wiced_tls_init_root_ca_certificates | 31852 | 29148 | 2704 | 2704 | 4096 | 2248 | |
###### mqtt_network_init / wiced_tls_init_identity | 40044 | 31740 | 8304 | 5256 | 8192 | 2592 | |
###### mqtt_network_init / wiced_tls_init_context | 40044 | 31740 | 8304 | 5256 | 0 | 0 | |
###### mqtt_network_init / wiced_tcp_enable_tls | 40044 | 31740 | 8304 | 5256 | 0 | 0 | |
###### mqtt_network_init / mqtt_network_connect | 44140 | 34844 | 9296 | 4736 | 4096 | 3104 | |
###### mqtt_network_init / wiced_tcp_register_callbacks | 44140 | 34844 | 9296 | 4736 | 0 | 0 | |
MQTT connection opened | 44140 | 34844 | 9296 | 4736 | 0 | 0 | |
###### mqtt_network_deinit / mqtt_network_deinit | 44140 | 34844 | 9296 | 4736 | 0 | 0 | |
###### mqtt_network_deinit / wiced_tcp_unregister_callbacks | 44140 | 34844 | 9296 | 4736 | 0 | 0 | |
###### mqtt_network_deinit / mqtt_network_disconnect | 44140 | 34844 | 9296 | 4736 | 0 | 0 | |
###### mqtt_network_deinit / wiced_tls_reset_context | 44140 | 32788 | 11352 | 4736 | 0 | -2056 | |
###### mqtt_network_deinit / wiced_tls_deinit_root_ca_certificates | 44140 | 30020 | 14120 | 8128 | 0 | -2768 | |
###### mqtt_network_deinit / wiced_tcp_delete_socket | 44140 | 30020 | 14120 | 8128 | 0 | 0 | |
MQTT connection closed | 44140 | 30020 | 14120 | 8128 | 0 | 0 | -3120 |
wiced_hostname_lookup | 44140 | 30020 | 14120 | 8128 | 0 | 0 | |
###### mqtt_network_init / mqtt_network_init | 44140 | 30020 | 14120 | 8128 | 0 | 0 | |
###### mqtt_network_init / wiced_tcp_create_socket | 44140 | 30020 | 14120 | 8128 | 0 | 0 | |
###### mqtt_network_init / wiced_tls_init_root_ca_certificates | 44140 | 32268 | 11872 | 8128 | 0 | 2248 | |
###### mqtt_network_init / wiced_tls_init_identity | 44140 | 34860 | 9280 | 6016 | 0 | 2592 | |
###### mqtt_network_init / wiced_tls_init_context | 44140 | 34860 | 9280 | 6016 | 0 | 0 | |
###### mqtt_network_init / wiced_tcp_enable_tls | 44140 | 34860 | 9280 | 6016 | 0 | 0 | |
###### mqtt_network_init / mqtt_network_connect | 48236 | 37972 | 10264 | 5496 | 4096 | 3112 | |
###### mqtt_network_init / wiced_tcp_register_callbacks | 48236 | 37972 | 10264 | 5496 | 0 | 0 | |
MQTT connection opened | 48236 | 37972 | 10264 | 5496 | 0 | 0 | |
###### mqtt_network_deinit / mqtt_network_deinit | 48236 | 37972 | 10264 | 5496 | 0 | 0 | |
###### mqtt_network_deinit / wiced_tcp_unregister_callbacks | 48236 | 37972 | 10264 | 5496 | 0 | 0 | |
###### mqtt_network_deinit / mqtt_network_disconnect | 48236 | 37972 | 10264 | 5496 | 0 | 0 | |
###### mqtt_network_deinit / wiced_tls_reset_context | 48236 | 35908 | 12328 | 5496 | 0 | -2064 | |
###### mqtt_network_deinit / wiced_tls_deinit_root_ca_certificates | 48236 | 33140 | 15096 | 8888 | 0 | -2768 | |
###### mqtt_network_deinit / wiced_tcp_delete_socket | 48236 | 33140 | 15096 | 8888 | 0 | 0 | |
MQTT connection closed | 48236 | 33140 | 15096 | 8888 | 0 | 0 | -3120 |
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hi Cypress,
The results of further investigation:
wiced_mqtt_connect -> mqtt_connection_init -> mqtt_network_init -> mqtt_network_connect -> wiced_tcp_connect (NetX_Duo) -> wiced_tcp_start_tls -> wiced_generic_start_tls_with_ciphers -> ssl_handshake_client_async
ssl_handshake_client_async( &tls_context->context ) allocates memory block but ssl_free(&tls_context->context) releases smaller memory block = MEMORY LEAK !
Beacuse SSL library is closed type, we can not correct that bug and another problems. It seems that BESL memory leak problems has long story (about 2 years) and up to now still not solved!
When you will provide useful BESL library?
Robert
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hi -
With a connected IP interface, we are calling the following to connect to aws iot:
wiced_mqtt_connect(mqtt_obj, address, interface, callbacks, security,
&conninfo);
Then waiting some time and calling,
wiced_mqtt_disconnect(mqtt_obj);
and then after a little more time
wiced_mqtt_deinit(mqtt_object);
Standard stuff when compared to the secure_mqtt app. I'm also ussing malloc_debug to note the allocated heap elements. After this process, here's what I see:
---- ----
bignum 312 20034810
bignum 312 200346d0
bignum 180 20034fe8
bignum 180 20034f30
bignum 180 20034e78
bignum 180 20034dc0
bignum 180 20033828
bignum 308 20034c88
bignum 56 200337e8
bignum 308 20034b50
tls 72 20033f18
bignum 56 20033ed8
bignum 308 20033da0
pubkey 212 20033cc8
x509 940 20033918
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hi,
We were using SDK_3.5.2 in which MQTT reconnect has issues.
Based on the discussion in the forum we ported our application SDK_4.1.0 but the memory leak issues exist.
I am explaining the application logic on a connect, disconnect and reconnect. Please let me know if there is something which we may be missing.
Connect:
1. Shadow_init() does aws_app_init() subscribing to channels
2. aws_mqtt_conn_open() ->
3.resource_get_readonly_buffer() which does read the certificates ->
4. wiced_mqtt_connect()
Disconnect: On WICED_MQTT_EVENT_TYPE_DISCONNECTED event from mqtt_connection_event_cb()
1. Shadow_close() ->
mqtt_network_deinit(&(((mqtt_connection_t*)app_info.mqtt_object)->socket));
mqtt_connection_deinit((mqtt_connection_t*) app_info.mqtt_object);
2. Then call the shadow_init to trigger aws_mqtt_conn_open()
In this scenario we observe that reconnect device hangs
Disconnect: On WICED_MQTT_EVENT_TYPE_DISCONNECTED event from mqtt_connection_event_cb()
1. Call the shadow_init to trigger aws_mqtt_conn_open()
In this scenario we observe that reconnect fails with error code (3036)
Looking forward for quick response.
Thanks in advance.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
tested with WICED 5.0. Closing MQTT connection no longer leaves the BESL items on the heap. Issue appears resolved.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Thanks Dave.