Unfortunately, the Doxygen generated HTML based files that are included within the SDK are the only API documentation we offer.
The sample applications within the SDK are documented fairly well and many users find those applications helpful for getting started.
The RSA library is implemented in firmware on the 20737 and is a direct port of the Polar SSL library (version 1.7).
I believe the same library can be obtained (in source) from http://www.polarssl.org(or licensed from them if needed).
What little we have in the way of documentation for this feature is available within the WICED-Smart\bleapp\utils\rsa.h file.
The following sample application leverages the RSA library, so it may be a good place to start: /WICED-Smart-SDK/Apps/ota_secure_firmware_upgrade
Thank you for the information. I will check out the resources you indicated.