I am currently using the following HighCharts:HighStock:Charts: http://www.highcharts.com/stock/demo/data-grouping in order to display the data returned from the server.

We retrieve the data from a MySQL database and is really big. We are storing sensor metrics every 1 second. After a while we got the following error:

[Wed Sep 12 00:15:56 2012] [error] [client 127.0.0.1] PHP Fatal error:  Allowed memory size of 134217728 bytes exhausted (tried to allocate 4756882 bytes) in C:\\wamp\\www\\admin\\getTrends.php on line 156, referer: http://localhost/admin/trends.php
[Wed Sep 12 00:15:56 2012] [error] [client 127.0.0.1] PHP Stack trace:, referer: http://localhost/admin/trends.php
[Wed Sep 12 00:15:56 2012] [error] [client 127.0.0.1] PHP   1. {main}() C:\\wamp\\www\\admin\\getTrends.php:0, referer: http://localhost/admin/trends.php
[Wed Sep 12 00:15:56 2012] [error] [client 127.0.0.1] PHP   2. getTrendsDataAI() C:\\wamp\\www\\admin\\getTrends.php:33, referer: http://localhost/admin/trends.php
[Wed Sep 12 00:15:56 2012] [error] [client 127.0.0.1] PHP   3. printResults() C:\\wamp\\www\\admin\\getTrends.php:102, referer: http://localhost/admin/trends.php
[Wed Sep 12 00:15:56 2012] [error] [client 127.0.0.1] PHP   4. createData() C:\\wamp\\www\\admin\\getTrends.php:230, referer: http://localhost/admin/trends.php
[Wed Sep 12 00:15:56 2012] [error] [client 127.0.0.1] PHP   5. implode() C:\\wamp\\www\\admin\\getTrends.php:156, referer: http://localhost/admin/trends.php

What is the best solution to return this data as JSON object to HighStocks for viewing? And how can we overcome the PHP limitation? Shall we return chunk of data each time? How do they usually present enormous amount of data to the users and creating charts and reports from this data? Another big problem that we need to overcome is that the returned JSON object is enormous. At this point is around 20-30 mbs and it will be much larger in the future. Is it ok to return this data to the user and perform everything client side?

Any suggestions or thoughts welcome.