Memory leak phenomenon occurs when calling MQTT API repeatedly

Tip / Sign in to post questions, reply, level up, and achieve exciting badges. Know more

cross mob
KEKA_4568351
Level 4
Level 4
50 replies posted 25 replies posted 10 replies posted

<English>

Repeated calls to the MQTT API have resulted in memory leaks.

"◆ MQTT API processing procedure" is as follows.

The processing procedure starts from "Referenced sample code".

In the developed application, there is no process to dynamically generate the heap area such as the malloc function, so I don't think it is a problem in the application.

I think the "wiced_aws_create_endpoint" function is the problem.

In the "wiced_aws_create_endpoint" function, the aws_internal_mqtt_init function is called, but there is no place where the aws_internal_mqtt_deinit function is called, and it is thought that there is a memory leak.

Refer to "◆Wiced_aws_create_endpoint function, Malloc, queue, create_thread function"

The size of the memory leak is "8,672" bytes, but the total of the three functions of "◆Wiced_aws_create_endpoint function, Malloc, queue, create_thread function" is "8,626" bytes, which is an approximate value. Is out.

Please let me know if the "◆ MQTT API processing procedure" is bad or if there are any other countermeasures.

◆ MQTT API processing procedure

・ Wiced_aws_init

・ Wiced_aws_create_endpoint ★

・ Wiced_aws_connect

・ Wiced_aws_publish / wiced_aws_subscribe

・ Wiced_aws_disconnect

・ Wiced_aws_deinit

◆ Referenced sample code

・ WICED-Studio-6.4 \ 43xxx_Wi-Fi \ apps \ demo \ aws \ iot \ pub_sub \ publisher \ publisher.c

・ WICED-Studio-6.4 \ 43xxx_Wi-Fi \ apps \ demo \ aws \ iot \ pub_sub \ subscriber \ subscriber.c

◆ Environment

・ Version: WICED SDK with PSoC 6 Support 6.4.0

・ Project: 43xxx_Wi-Fi

◆ Wiced_aws_create_endpoint function, Malloc, queue, create_thread function

・ Aws_internal_mqtt_init: 1864 Byte

・ Wiced_rtos_init_queue: 640 Byte

・ Mqtt_core_init: 6122 Byte

◆ aws_internal_mqtt_deinit function Grep result

・ WICED-Studio-6.4 \ 43xxx_Wi-Fi \ libraries \ protocols \ AWS \ aws_mqtt.c (345,16) [SJIS]: wiced_result_t aws_internal_mqtt_deinit (wiced_aws_internal_handle_t * aws)

・ WICED-Studio-6.4 \ 43xxx_Wi-Fi \ libraries \ protocols \ AWS \ aws_internal.h (132,16) [SJIS]: wiced_result_t aws_internal_mqtt_deinit (wiced_aws_internal_handle_t * aws);

>> No function is called

<Japanese>

MQTT APIを繰り返しコールすると、メモリリークの現象が発生する

MQTT APIを繰り返しコールすると、メモリリークの現象が発生しています。

「◆MQTT APIの処理手順」は、以下のとおりです。

「参考にしたサンプルコード」からの処理手順になります。

開発したアプリケーションでは、malloc関数などのヒープ領域を動的に生成する処理がないため、アプリケーションでの問題ではないと思っています。

wiced_aws_create_endpoint関数が、問題だと思っています。

wiced_aws_create_endpoint関数内で、aws_internal_mqtt_init関数がコールされていますが、aws_internal_mqtt_deinit関数をコールしている箇所がなく、メモリリークしていると考えています。

「◆wiced_aws_create_endpoint関数で、Malloc、queue、create_threadをしている関数」参照

メモリリークしていると思うサイズが、「8,672」バイトですが、「◆wiced_aws_create_endpoint関数で、Malloc、queue、create_threadをしている関数」の3関数のトータルが、「8,626」バイトであり、近似値がでています。

「◆MQTT APIの処理手順」が悪いのか、他に対応策がありましたら、教えてください。

◆MQTT APIの処理手順

・wiced_aws_init

・wiced_aws_create_endpoint  ★

・wiced_aws_connect

・wiced_aws_publish/wiced_aws_subscribe

・wiced_aws_disconnect

・wiced_aws_deinit

◆参考にしたサンプルコード

・WICED-Studio-6.4\43xxx_Wi-Fi\apps\demo\aws\iot\pub_sub\publisher\publisher.c

・WICED-Studio-6.4\43xxx_Wi-Fi\apps\demo\aws\iot\pub_sub\subscriber\subscriber.c

◆環境

・Version:WICED SDK with PSoC 6 Support 6.4.0

・Project:43xxx_Wi-Fi

◆wiced_aws_create_endpoint関数で、Malloc、queue、create_threadをしている関数

・aws_internal_mqtt_init:1864 Byte

・wiced_rtos_init_queue:640 Byte

・mqtt_core_init:6122 Byte

◆aws_internal_mqtt_deinit関数 Grep結果

・WICED-Studio-6.4\43xxx_Wi-Fi\libraries\protocols\AWS\aws_mqtt.c(345,16)  [SJIS]: wiced_result_t aws_internal_mqtt_deinit( wiced_aws_internal_handle_t* aws )

・WICED-Studio-6.4\43xxx_Wi-Fi\libraries\protocols\AWS\aws_internal.h(132,16)  [SJIS]: wiced_result_t aws_internal_mqtt_deinit( wiced_aws_internal_handle_t* aws );

>>コールしている関数がない

0 Likes
1 Solution

In this memory leak, two problems were found.

Please comment if there is any problem as the modified code will be deployed.

1. Memory leak in Wiced_aws_create_endpoint function

   By adding the following “★ Additional code ★”, the heap area of the Wiced_aws_create_endpoint function is released.

◆MQTT disconnect Additional code

==========

if (0 != g_aws_connection) {

    //↓↓↓★Additional code★↓↓↓

    p_aws = (wiced_aws_internal_handle_t*)g_aws_connection;

    mqtt_object = p_aws->mqtt.base;

    if(NULL != mqtt_object) {

        p_mqtt_conn = (mqtt_connection_t*)mqtt_object;

    }

    //↑↑↑★Additional code★↑↑↑

    /* AWS disconnect */

    wiced_result = wiced_aws_disconnect(g_aws_connection);

    /* semaphore deinit */

    wiced_result = wiced_rtos_deinit_semaphore(&s_event_semaphore);

    //↓↓↓★Additional code★↓↓↓

    if(NULL != p_mqtt_conn) {

        wiced_result = mqtt_core_deinit(p_mqtt_conn);

    }

    if(NULL != mqtt_object) {

        free(mqtt_object);

    }

    //↑↑↑★Additional code★↑↑↑

}

wiced_result = wiced_aws_deinit();

==========

2. Memory leak in wiced_aws_connect function

   aws_internal_mqtt_connect of the wiced_aws_connect function >> In the wiced_mqtt_connect function, when the connection fails with the mqtt_connect function, there is no release of the heap area secured by the mqtt_connection_init function, resulting in a memory leak.

   The following "★ Update code ★" has been added to release the heap area of the mqtt_connection_init function.

◆wiced_mqtt_connect function

==========

wiced_result_t wiced_mqtt_connect( wiced_mqtt_object_t mqtt_obj, wiced_ip_address_t *address, wiced_interface_t interface, wiced_mqtt_callback_t callback, wiced_mqtt_security_t *security, wiced_mqtt_pkt_connect_t *conninfo )

{

・・・

    result = mqtt_connection_init( address, conninfo->port_number, interface, callback, conn, security );

・・・

#if 0

//↓↓↓★Memory Leak★↓↓↓

// If there is an error in the mqtt_connect function, the heap area allocated by the mqtt_connection_init function will not be released.

    return mqtt_connect( conn, &args, &session );

//↑↑↑★Memory Leak★↑↑↑

#endif

#if 1

    //↓↓↓★Update code★↓↓↓

    result = mqtt_connect( conn, &args, &session );

    if ( result != WICED_SUCCESS )

    {

        mqtt_connection_deinit(conn);

        WPRINT_LIB_INFO(("[MQTT LIB] : mqtt_connect error - mqtt_connection_deinit! \n"));

    }

    return result;

    //↑↑↑★Update code★↑↑↑

#endif

}

==========

<Japanese>

今回のメモリリークは、大きく2点の問題が発見されました。

修正コードを展開しますので、問題がないかコメントをお願いします。

1.Wiced_aws_create_endpoint関数のメモリーリーク

  以下の「★Additional code★」の追加により、Wiced_aws_create_endpoint関数のヒープ領域の解放をおこなっています。

◆MQTT disconnect Additional code

==========

if (0 != g_aws_connection) {

    //↓↓↓★Additional code★↓↓↓

    p_aws = (wiced_aws_internal_handle_t*)g_aws_connection;

    mqtt_object = p_aws->mqtt.base;

    if(NULL != mqtt_object) {

        p_mqtt_conn = (mqtt_connection_t*)mqtt_object;

    }

    //↑↑↑★Additional code★↑↑↑

    /* AWS disconnect */

    wiced_result = wiced_aws_disconnect(g_aws_connection);

    /* semaphore deinit */

    wiced_result = wiced_rtos_deinit_semaphore(&s_event_semaphore);

    //↓↓↓★Additional code★↓↓↓

    if(NULL != p_mqtt_conn) {

        wiced_result = mqtt_core_deinit(p_mqtt_conn);

    }

    if(NULL != mqtt_object) {

        free(mqtt_object);

    }

    //↑↑↑★Additional code★↑↑↑

}

wiced_result = wiced_aws_deinit();

==========

2.wiced_aws_connect関数のメモリーリーク

  wiced_aws_connect関数のaws_internal_mqtt_connect >> wiced_mqtt_connect関数で、mqtt_connect関数で接続失敗時に、mqtt_connection_init関数で確保したヒープ領域の解放がないため、メモリリークしている。

  以下の「★Update code★」の追加により、mqtt_connection_init関数のヒープ領域の解放をおこなっています。

◆wiced_mqtt_connect関数

==========

wiced_result_t wiced_mqtt_connect( wiced_mqtt_object_t mqtt_obj, wiced_ip_address_t *address, wiced_interface_t interface, wiced_mqtt_callback_t callback, wiced_mqtt_security_t *security, wiced_mqtt_pkt_connect_t *conninfo )

{

・・・

    result = mqtt_connection_init( address, conninfo->port_number, interface, callback, conn, security );

・・・

#if 0

//↓↓↓★Memory Leak★↓↓↓

// mqtt_connect関数にエラーがある場合、mqtt_connection_init関数で確保したヒープ領域が解放されません。

    return mqtt_connect( conn, &args, &session );

//↑↑↑★Memory Leak★↑↑↑

#endif

#if 1

    //↓↓↓★Update code★↓↓↓

    result = mqtt_connect( conn, &args, &session );

    if ( result != WICED_SUCCESS )

    {

        mqtt_connection_deinit(conn);

        WPRINT_LIB_INFO(("[MQTT LIB] : mqtt_connect error - mqtt_connection_deinit! \n"));

    }

    return result;

    //↑↑↑★Update code★↑↑↑

#endif

}

View solution in original post

0 Likes
10 Replies
RaktimR_11
Moderator
Moderator
Moderator
500 replies posted 250 replies posted 100 replies posted

Did you check the memory leak with mallinfo, if yes can you please provide the log?

0 Likes
lock attach
Attachments are accessible only for community members.

Thank you very much.

Attach the mallinfo log.

0 Likes

Can you solve it?

What is your current status?

Please let me know if there are any alternatives.

<Japanese>

解決できそうですか?

現在のステータスを教えてください。

代替案があれば、教えてください。

0 Likes

Please let me know if the usage is wrong or a bug.

0 Likes

I would recommend you to check once with this application: CypressAcademy_WW101_Files/Projects/ww101key/07c/04_publisher_mqtt at master · cypresssemiconductorc...

I usually use my publisher app which looks somewhat like this and with this app, I don't see the memleak issue. The default demo application is not correctly freeing up the memory in my opinion. Instead, if you would be so kind to check once with this application, we can converge on one thing and investigate the default demo app further.

Sincerely apologize for the delay in responding to this thread.

thank you for your answer.

As a module is released, it is not realistic to change from wiced_aws_XXX functions to wiced_mqtt_XXX functions.

The wiced_aws_XXX functions need an alternative to free memory.

It is OK to modify the source code of WICED SDK.

<Japanese>

回答ありがとうございます。

モジュールとしては、リリースしているものであるため、wiced_aws_XXX関数からwiced_mqtt_XXX関数に変更するのは、現実的ではありません。

wiced_aws_XXX関数で、メモリ解放する代替案が必要としています。

WICED SDKのソース修正でも、構わないので、代替案をお願いします。

0 Likes

In this memory leak, two problems were found.

Please comment if there is any problem as the modified code will be deployed.

1. Memory leak in Wiced_aws_create_endpoint function

   By adding the following “★ Additional code ★”, the heap area of the Wiced_aws_create_endpoint function is released.

◆MQTT disconnect Additional code

==========

if (0 != g_aws_connection) {

    //↓↓↓★Additional code★↓↓↓

    p_aws = (wiced_aws_internal_handle_t*)g_aws_connection;

    mqtt_object = p_aws->mqtt.base;

    if(NULL != mqtt_object) {

        p_mqtt_conn = (mqtt_connection_t*)mqtt_object;

    }

    //↑↑↑★Additional code★↑↑↑

    /* AWS disconnect */

    wiced_result = wiced_aws_disconnect(g_aws_connection);

    /* semaphore deinit */

    wiced_result = wiced_rtos_deinit_semaphore(&s_event_semaphore);

    //↓↓↓★Additional code★↓↓↓

    if(NULL != p_mqtt_conn) {

        wiced_result = mqtt_core_deinit(p_mqtt_conn);

    }

    if(NULL != mqtt_object) {

        free(mqtt_object);

    }

    //↑↑↑★Additional code★↑↑↑

}

wiced_result = wiced_aws_deinit();

==========

2. Memory leak in wiced_aws_connect function

   aws_internal_mqtt_connect of the wiced_aws_connect function >> In the wiced_mqtt_connect function, when the connection fails with the mqtt_connect function, there is no release of the heap area secured by the mqtt_connection_init function, resulting in a memory leak.

   The following "★ Update code ★" has been added to release the heap area of the mqtt_connection_init function.

◆wiced_mqtt_connect function

==========

wiced_result_t wiced_mqtt_connect( wiced_mqtt_object_t mqtt_obj, wiced_ip_address_t *address, wiced_interface_t interface, wiced_mqtt_callback_t callback, wiced_mqtt_security_t *security, wiced_mqtt_pkt_connect_t *conninfo )

{

・・・

    result = mqtt_connection_init( address, conninfo->port_number, interface, callback, conn, security );

・・・

#if 0

//↓↓↓★Memory Leak★↓↓↓

// If there is an error in the mqtt_connect function, the heap area allocated by the mqtt_connection_init function will not be released.

    return mqtt_connect( conn, &args, &session );

//↑↑↑★Memory Leak★↑↑↑

#endif

#if 1

    //↓↓↓★Update code★↓↓↓

    result = mqtt_connect( conn, &args, &session );

    if ( result != WICED_SUCCESS )

    {

        mqtt_connection_deinit(conn);

        WPRINT_LIB_INFO(("[MQTT LIB] : mqtt_connect error - mqtt_connection_deinit! \n"));

    }

    return result;

    //↑↑↑★Update code★↑↑↑

#endif

}

==========

<Japanese>

今回のメモリリークは、大きく2点の問題が発見されました。

修正コードを展開しますので、問題がないかコメントをお願いします。

1.Wiced_aws_create_endpoint関数のメモリーリーク

  以下の「★Additional code★」の追加により、Wiced_aws_create_endpoint関数のヒープ領域の解放をおこなっています。

◆MQTT disconnect Additional code

==========

if (0 != g_aws_connection) {

    //↓↓↓★Additional code★↓↓↓

    p_aws = (wiced_aws_internal_handle_t*)g_aws_connection;

    mqtt_object = p_aws->mqtt.base;

    if(NULL != mqtt_object) {

        p_mqtt_conn = (mqtt_connection_t*)mqtt_object;

    }

    //↑↑↑★Additional code★↑↑↑

    /* AWS disconnect */

    wiced_result = wiced_aws_disconnect(g_aws_connection);

    /* semaphore deinit */

    wiced_result = wiced_rtos_deinit_semaphore(&s_event_semaphore);

    //↓↓↓★Additional code★↓↓↓

    if(NULL != p_mqtt_conn) {

        wiced_result = mqtt_core_deinit(p_mqtt_conn);

    }

    if(NULL != mqtt_object) {

        free(mqtt_object);

    }

    //↑↑↑★Additional code★↑↑↑

}

wiced_result = wiced_aws_deinit();

==========

2.wiced_aws_connect関数のメモリーリーク

  wiced_aws_connect関数のaws_internal_mqtt_connect >> wiced_mqtt_connect関数で、mqtt_connect関数で接続失敗時に、mqtt_connection_init関数で確保したヒープ領域の解放がないため、メモリリークしている。

  以下の「★Update code★」の追加により、mqtt_connection_init関数のヒープ領域の解放をおこなっています。

◆wiced_mqtt_connect関数

==========

wiced_result_t wiced_mqtt_connect( wiced_mqtt_object_t mqtt_obj, wiced_ip_address_t *address, wiced_interface_t interface, wiced_mqtt_callback_t callback, wiced_mqtt_security_t *security, wiced_mqtt_pkt_connect_t *conninfo )

{

・・・

    result = mqtt_connection_init( address, conninfo->port_number, interface, callback, conn, security );

・・・

#if 0

//↓↓↓★Memory Leak★↓↓↓

// mqtt_connect関数にエラーがある場合、mqtt_connection_init関数で確保したヒープ領域が解放されません。

    return mqtt_connect( conn, &args, &session );

//↑↑↑★Memory Leak★↑↑↑

#endif

#if 1

    //↓↓↓★Update code★↓↓↓

    result = mqtt_connect( conn, &args, &session );

    if ( result != WICED_SUCCESS )

    {

        mqtt_connection_deinit(conn);

        WPRINT_LIB_INFO(("[MQTT LIB] : mqtt_connect error - mqtt_connection_deinit! \n"));

    }

    return result;

    //↑↑↑★Update code★↑↑↑

#endif

}

0 Likes

Thank you for sharing your changes. In my opinion, these were the areas where the heap memory was not correctly getting freed if you use the default demo application. It's okay for you to go ahead with these changes and deploy the modified code. Meanwhile, I will file a bug report to our internal software team so that the mem leak is correctly handled.

Thank you very much.

There are two confirmations.

[QA]

・ When will the bug be fixed? (Is it unknown?)

・ If there is any other code that may cause a memory leak related to the WICED_AWS_XXX functions, please let us know.

The WICED_AWS_XXX functions feel that there is a quality problem.

<japanese>

ありがとうございます。

2点確認があります。

【QA】

・バグの修正は、いつごろになりますか。(不明ですか?)

・WICED_AWS_XXX関数の関連で、他にメモリリークの可能性のあるコードがあれば、教えてください。

WICED_AWS_XXX関数は、品質に問題があると感じます。

0 Likes

I have reported the same to our internal development team. I don't have a timeline for the bug fixes yet. Once I have that, i will update the thread. I assure you that the WICED_AWS_XXX functions pass through stringent quality checks before it is made available for the public. I will pass this feedback as well and want to point out one additional thing. The libraries are well designed as always but the demo application is not handling the mem alloc well which is what I am going to test further.