9 Replies Latest reply on Jun 15, 2019 6:06 PM by user_13463998

    store measured values in CSV and output chart - PSOC 5LP

    Jannick_Gegenheimer_4104756

      Hello everybody,

      I am still a noobie here and hope that you can help me :-).

      I have written a program in which I determine a voltage drop (ADC converter) via a shunt resistor, then calculate the current for me and then output the measured value via HTerm.

      Now I would like to have the readings displayed in a stream-time diagram and, optimally, store the values in a .csv or .xls.

      I would like it if the Excel list looks like this:

       

      DateTimeVoltage Ush [mV]Current Ipv [mA}
      14.06.201911:31:45402,7862346,7130771
      14.06.2019

      11:32:22

      403,3246786,7220780
      14.06.201911:33:01405,1242556,7520709

       

      The final version would be to average every minute over a measurement and then every five minutes to average the previous average. So that a value is output every 5 minutes.

      So should be the diagram that every 5 minutes a new value is displayed.

       


      Thank you very much and sunny greetings from Germany
      jg.vs

        • 1. Re: store measured values in CSV and output chart - PSOC 5LP
          user_13463998

          Hi,

           

          Sometime ago, I posted following topic.

          How I recycle the data (aka Return of CCS811)

           

          Can this be a hint for you?

           

          moto

          • 2. Re: store measured values in CSV and output chart - PSOC 5LP
            KTrenholm_1955226

            I usually use RealTerm for serial logging like this, since it will automatically Timestamp your data with different choices for formatting and delimiter (comma or space) and write out to a text file log.  I may have to give TeraTerm a try though.  SerialPlot also looks like it could be quite handy.

             

            In any case, you can just send your data via serial, comma delimited, with a Carriage Return/Line Feed at the end of each line every 5 minutes (or whatever time period you choose).  All you need then is a terminal that will log your data to a file.  You may also want to print a header row before you start sending values to give your columns names, otherwise you'll have to do it manually afterward.  At the end you will have a nice comma separated text file that you can just import into spreadsheet software.

            • 3. Re: store measured values in CSV and output chart - PSOC 5LP
              user_342122993

              Jannick,

              I typically use a Multichart software, it is old charting tool from Cypress. It allows simultaneous input of 3 data streams using serial connection. You can find the Multicart software and some demo projects here:

              (in #2)

              Re: Best Approach: Implement DMA on 16bit Timer or 2 Status Registers?

              (in #3)

              Re: PSoC Today! - Synchronous Detection Detail

               

              /odissey1

              • 4. Re: store measured values in CSV and output chart - PSOC 5LP
                Jannick_Gegenheimer_4104756

                Hello everybody,

                Thank you for the answers.

                Unfortunately, I am not quite so clear, I would like to get rid of more detailed information to describe my problem in more detail:



                1. I tried to get the current date and time, code:

                UART_1_PutString ("Date:%s Time:%s) \ n", __DATE__, __TIME__);


                the compiler already shows the error: too many arguments to function call, expected single argument 'string', have 3 arguments

                Where is my mistake?



                2. My program should automatically create a csv file that stores every 5 minutes my reading, which I submit via UART.
                So program runs after debugging in an infinite loop and sends every 5 minutes a reading.
                At the same time, the measured values should also be displayed in a graphic.
                Like a kind of monitoring, eg. online I can look at the graph of my readings and export the data also in a csv.

                 



                Thank you very much for your help.

                greetings
                jg.vs

                • 5. Re: store measured values in CSV and output chart - PSOC 5LP
                  user_119654

                  jg.vs,

                   

                  1.  PutString() only takes a single string for an argument.  What you need to add is the following:

                  ...  // other lines of #includes.

                  #include <stdio.h>

                  ...  // other lines of code.

                  #define TSTR_SZ  100  // set this number to the maximum string size you expect to need.

                  char8 tstr[TSTR_SZ];

                  ...  // other lines of code.

                  ... function_call(...)    // Your function

                  {

                  ...  // other lines of code.

                  snprintf(tstr, sizeof(tstr), "Date:%s Time:%s) \ n", __DATE__, __TIME__);    // format the string with the variables needed

                  UART_1_PutString (tstr);          //  This will send the formatted string tstr out the UART port.

                  ...  // rest of code.

                   

                  One suggestion:  If you format your UART text output with commas ',' separating values on each line, you can usually use the log function of most terminal programs to store as .txt or as .csv.   Then as suggested earlier, you can use the charting function in Excel to display the values graphically.

                   

                  2. I've never used MultiChart. Looks useful.   You can trust /odissey1 (user_342122993).  He is a seasoned PSoC user and has very creative ideas.

                   

                  Len

                  • 6. Re: store measured values in CSV and output chart - PSOC 5LP
                    user_342122993

                    jg.vs,

                    It looks like first ")" is unnecessary

                    snprintf(tstr, sizeof(tstr), "Date:%s Time:%s) \ n", __DATE__, __TIME__); 

                     

                    should be:

                    snprintf(tstr, sizeof(tstr), "Date:%s Time:%s \n", __DATE__, __TIME__); 

                     

                    As Motoo Tanaka pointed out, PSoC5 does not generate real time by itself. You would need some external time keeper like DS3231. Better solution was proposed by KTrenholm, as to use timestamp on the receiving side, which is available by default. So, after all, you won't need time formatting at all.

                    /odissey1

                    • 7. Re: store measured values in CSV and output chart - PSOC 5LP
                      user_342122993

                      jg.vs,

                      2. The Multichart allows for charting/logging of the fast data streams (<10kHz), which may be overkill for your application. But there are many other options to plot/save slow data (<100Hz) using android devices and Bluetooth. For example:

                      Visual Logger

                      VisualLogger (Terminal/Graph) - Apps on Google Play

                      Screenshot Image

                       

                      BT Graphics

                      BT Terminal/Graphics Full - Apps on Google Play

                      Screenshot Image

                       

                      In this case you have to attach some cheap BT dongle like HC-05 to PSoC UART, and have your data on the phone or tablet.

                      /odissey1

                      1 of 1 people found this helpful
                      • 8. Re: store measured values in CSV and output chart - PSOC 5LP
                        user_13463998

                        Hi,

                         

                        > 1. I tried to get the current date and time, code:

                         

                        The "__DATE__" and "__TIME__" gives the date and time of when the program was compiled.

                        So it won't change every time you use it, in other words, these does not give you the "current time"

                        To get current time, off my head there are (at least) two methods,

                        (1) Use "RTC" component, but it requires an external 32KHz Crystal.

                        (2) Hand craft  or find unix time calendar code and keep time using some timer,

                            this is a lot of work and very poor timer accuracy.

                        So may be, as others suggest using PC or Host side time utility is much cheaper and accurate.

                         

                        Having written that the syntax error will be fixed if you write

                          UART_PutString(__DATE__) ;

                          UART_PutString(" ") ;

                          UART_PutString(__TIME__) ;

                          UART_PutString("\n") ;

                        or use sprintf, snprintf to compose a string then use UART_PutString(str)

                         

                        > 2. My program should automatically create a csv file that stores every 5 minutes my reading, which I submit via UART.

                        So program runs after debugging in an infinite loop and sends every 5 minutes a reading.

                        At the same time, the measured values should also be displayed in a graphic.

                        Like a kind of monitoring, eg. online I can look at the graph of my readings and export the data also in a csv.

                         

                        IMHO, this is something you need to think and implement or pay someone to implement it for you.

                        Have you read/tried my sample at the URL I posted earlier?

                         

                        > Sometime ago, I posted following topic.

                        > How I recycle the data (aka Return of CCS811)

                         

                        Although not too fancy, I think it is showing one possible method to fulfill your requirement.

                        (1) Showing real time graphics

                        (2) Storing csv on your PC

                        And also those suggestions from others should be help.

                         

                        Anyway, happy hacking ;-)

                         

                        moto

                        1 of 1 people found this helpful
                        • 9. Re: store measured values in CSV and output chart - PSOC 5LP
                          user_13463998

                          Well, after the previous post of mine, I found the following URL

                          https://www.oryx-embedded.com/doc/date__time_8c_source.html

                           

                          So I tweaked a little,

                           

                          schematic

                          000-schematic.JPG

                           

                          main.c

                          For the fast result, I set Interval = 1, but for 5 minutes interval, please make it to 300,

                          Meantime if you have live RTC, you don't have to call set_date_time() function.

                          but for the realistic time, I added this function. (I was younger in 1970 ;-)

                          =========================

                          #include "project.h"

                          #include "stdio.h"

                          #include "date_time.h"

                           

                          #define STR_LEN 32

                          #define RX_BUF_LEN 128

                           

                          #define SPACE ' '

                          #define TAB '\t'

                          #define CR  '\r'

                          #define LF  '\n'

                           

                          volatile uint32_t unix_time = 0 ;

                          volatile int      pit_flag = 0 ;

                          volatile char    rx_buf[RX_BUF_LEN] ;

                          volatile int      rx_write_index = 0 ;

                          int              rx_read_index = 0 ;

                          char              str[STR_LEN+1] ; /* print buffer */

                          int              str_index = 0 ;

                           

                          inline int        is_delimiter(uint8_t c) ;

                          void              print(char *str) ;

                          void              init_hardware(void) ;

                          void              splash(void) ;

                          int              get_string(char *str) ;

                          void              prompt(void) ;

                           

                          int              year    = 2019 ;

                          int              month  = 05 ;

                          int              day    = 16 ;

                          int              hours  = 8;

                          int              minutes = 32;

                          int              seconds = 36 ;

                           

                          float            R_value = 4700 ; /* 4.7 k ohm for place holder */

                           

                          CY_ISR(uart_rx_isr)

                          {

                              uart_rx_int_ClearPending() ;

                              if (UART_GetRxBufferSize()) {

                                  rx_buf[rx_write_index] = UART_GetByte() ;

                                  rx_write_index = (rx_write_index + 1) % RX_BUF_LEN ;

                              }

                          }

                           

                          CY_ISR(timer_isr)

                          {

                              timer_int_ClearPending() ;

                              Timer_ReadStatusRegister() ;

                              unix_time++ ;

                              pit_flag = 1 ;

                          }

                           

                          void set_date_time(void)

                          {

                              DateTime date_time ;

                              print("Enter Date (YYYY/MM/DD) > ") ;

                              while(get_string(str) <= 0) ;

                              sscanf(str, "%d/%d/%d", &year, &month, &day) ;

                              print("Enter Time (hh:mm:ss) > ") ;

                              while(get_string(str) <= 0) ;

                              sscanf(str, "%d:%d:%d", &hours, &minutes, &seconds) ;

                              date_time.year    = year ;

                              date_time.month  = month - 1 ; /* unix time, month is 0~11 instead of 1~12 orz */

                              date_time.day    = day ;

                              date_time.hours  = hours ;

                              date_time.minutes = minutes ;

                              date_time.seconds = seconds ;

                              unix_time = convertDateToUnixTime(&date_time) ;

                          }

                           

                          void measure(float *mV, float *mA)

                          {

                              int16_t adc_count ;

                              ADC_StartConvert() ;

                              ADC_IsEndConversion(ADC_WAIT_FOR_RESULT) ;

                              adc_count = ADC_GetResult16(0) ; /* channel 0 */

                              *mV = (float)ADC_CountsTo_mVolts(adc_count) ;

                              *mA = *mV / R_value ;

                          }

                           

                          void print_csv_title(void)

                          {

                              print("Date, Time, Voltage Ush[mV], Current lpv[mA]\n") ;

                          }

                           

                          void print_csv(float mV, float mA)

                          {

                              DateTime date_time ;

                           

                              convertUnixTimeToDate(unix_time, &date_time) ;

                           

                              sprintf(str, "%d.%02d.%4d, ",

                                  date_time.day,

                                  date_time.month + 1,

                                  date_time.year) ;

                              print(str) ;

                            

                              sprintf(str, "%02d:%02d:%02d, ",

                                  date_time.hours,

                                  date_time.minutes,

                                  date_time.seconds ) ;

                              print(str) ;

                            

                              sprintf(str, "%d.%04d, ",

                                  (int)mV, (int)(mV * 10000) % 10000) ;

                              print(str) ;

                            

                              sprintf(str, "%d.%04d\n",

                                  (int)mA, (int)(mA * 10000) % 10000) ;

                              print(str) ;

                          }

                           

                          int main(void)

                          {

                              float mV, mA ;

                              uint32_t count = 0 ;

                              uint32_t interval = 1 ; //  300 ; /* 5 minutes */

                            

                              int period = 0 ;

                            

                              init_hardware() ;

                            

                              splash() ;

                            

                              sprintf(str, "Interval = %d sec\n", interval) ;

                              print(str) ;

                           

                              set_date_time() ;

                            

                              print_csv_title() ;

                              for(;;) {

                                  if (pit_flag) { /* timer interrupt occurred */

                                      pit_flag = 0 ;

                                      count++ ;

                                      if (count >= interval) {

                                          count = 0 ;

                                          measure(&mV, &mA) ;

                                          print_csv(mV, mA) ;

                                      }

                                  }

                              }

                          }

                           

                          inline int is_delimiter(uint8_t c)

                          {

                              int result = 0 ;

                              switch(c) {

                              case CR:

                              case LF:

                              case TAB:

                              case SPACE:

                                  result = c ;

                                  break ;

                              }

                              return( result ) ;

                          }

                           

                          void init_hardware(void)

                          {

                              UART_ClearRxBuffer() ;

                              uart_rx_int_ClearPending() ;

                              uart_rx_int_StartEx(uart_rx_isr) ;

                              UART_Start() ;

                            

                              timer_int_ClearPending() ;

                              timer_int_StartEx(timer_isr) ;

                              Timer_ReadStatusRegister() ;

                              Timer_Start() ;

                            

                              ADC_Start() ;

                            

                              CyGlobalIntEnable; /* Enable global interrupts. */

                          }

                           

                          void splash(void)

                          {

                              sprintf(str, "\nTimer Test (%s %s)\n", __DATE__, __TIME__) ;

                              print(str) ;

                          }

                           

                          void print(char *str)

                          {

                              UART_PutString(str) ;

                          }

                           

                          void prompt(void)

                          {

                              print("> ") ;

                          }

                           

                          int get_string(char str[])

                          {

                              int result = 0 ;

                              static int str_index = 0 ;

                            

                              if (rx_read_index != rx_write_index) {

                                  if (is_delimiter(rx_buf[rx_read_index])) {

                                      str[str_index] = 0 ;

                                      str_index = 0 ;

                                      result = 1 ;

                                  } else {

                                      str[str_index] = rx_buf[rx_read_index] ;

                                      str_index++ ;

                                      if (str_index >= STR_LEN) {

                                          str[STR_LEN] = 0 ;

                                          str_index = 0 ;

                                          result = -1 ;

                                      }

                                  }

                                  rx_read_index = (rx_read_index + 1) % RX_BUF_LEN ;

                              }  

                              return( result ) ;

                          }

                          ===============

                           

                          Tera Term log

                          000-tera_term.JPG

                           

                          Serial Plot Settings

                          After disconnecting Tera Term,

                           

                          Data Format

                          001-serial_plot_data_format.JPG

                           

                          Plot (Note: I unchecked date and time )

                          002-Plot.JPG

                           

                          The displayed result

                          003-serial-plot.JPG

                           

                          Getting log from Tera Term

                          After disconnecting Serial Plot

                          Select Menu: File > Log ...

                          004-tera_term-log.JPG

                          Specify log file

                          It can be anywhere in your system with any name, but I chose log_190616.csv

                          005-log_dialog.JPG

                          After logged a little I stopped Tera Term then located and opened the log file

                          006-open-csv.JPG

                           

                          Excel showed following (I adjusted the width of columns)

                          007-exec-view.JPG

                           

                          Attached is my sample project using CY8CKIT-059.

                           

                          moto

                          1 of 1 people found this helpful