Ceres Essentials 2

A little bit more information.
Let's create another node test.item2, and save some datapoints to it (or, we may just copy test/item/ directory).

export CERES_TREE=/tmp/storage/ceres
export NODE=test.item2

$ ll `ceres-tree-find $CERES_TREE $NODE --fspath`
total 12
-rw-r--r-- 1 tmp tmp 144 Jun 22 20:48 1371923150@10.slice
-rw-r--r-- 1 tmp tmp   8 Jun 22 20:52 1371923570@10.slice
-rw-r--r-- 1 tmp tmp  88 Jun 22 20:59 1371923870@10.slice

I can read data with ceres-node-read, but right now I'm more interested to know what exactly data are stored to each slice. I can check it with 'slicecat' tool:

$ slicecat /tmp/storage/ceres/test/item/1371923150@10.slice
[1371923150] Sat Jun 22 20:45:50 2013 1.0
[1371923160] Sat Jun 22 20:46:00 2013 3.0
[1371923170] Sat Jun 22 20:46:10 2013 4.0
[1371923180] Sat Jun 22 20:46:20 2013 6.0
[1371923190] Sat Jun 22 20:46:30 2013 7.0
[1371923200] Sat Jun 22 20:46:40 2013 9.0
[1371923210] Sat Jun 22 20:46:50 2013 10.0
[1371923220] Sat Jun 22 20:47:00 2013 nan
[1371923230] Sat Jun 22 20:47:10 2013 nan
[1371923240] Sat Jun 22 20:47:20 2013 nan
[1371923250] Sat Jun 22 20:47:30 2013 nan
[1371923260] Sat Jun 22 20:47:40 2013 nan
[1371923270] Sat Jun 22 20:47:50 2013 nan
[1371923280] Sat Jun 22 20:48:00 2013 nan
[1371923290] Sat Jun 22 20:48:10 2013 nan
[1371923300] Sat Jun 22 20:48:20 2013 nan
[1371923310] Sat Jun 22 20:48:30 2013 nan
[1371923320] Sat Jun 22 20:48:40 2013 11.0

$ slicecat `ceres-tree-find $CERES_TREE $NODE --fspath`/1371923570@10.slice
[1371923570] Sat Jun 22 20:52:50 2013 12.0

$ slicecat `ceres-tree-find $CERES_TREE $NODE --fspath`/1371923870@10.slice

[1371923870] Sat Jun 22 20:57:50 2013 13.0
[1371923880] Sat Jun 22 20:58:00 2013 14.0
[1371923890] Sat Jun 22 20:58:10 2013 15.0
[1371923900] Sat Jun 22 20:58:20 2013 nan
[1371923910] Sat Jun 22 20:58:30 2013 nan
[1371923920] Sat Jun 22 20:58:40 2013 16.0
[1371923930] Sat Jun 22 20:58:50 2013 nan
[1371923940] Sat Jun 22 20:59:00 2013 17.0
[1371923950] Sat Jun 22 20:59:10 2013 18.0
[1371923960] Sat Jun 22 20:59:20 2013 19.0
[1371923970] Sat Jun 22 20:59:30 2013 20.0

OK, great.
I can check run ceres-node-read for this interval (from begin of the first slice to the end of the last slice):

$ ceres-node-read --tree=$CERES_TREE $NODE  --fromtime=1371923150 --untiltime=$((1371923870+88/8*10))

and I see that it is almost identical, the only difference is that gaps are filled with None values.

Now, I'm going to change 'timeStep' for the first slice, from 10 to 60 (for not any reason, just for fun).

$ mv 1371923150@10.slice 1371923150@60.slice

$ slicecat `ceres-tree-find $CERES_TREE $NODE --fspath`/1371923150@60.slice 

[1371923150] Sat Jun 22 20:45:50 2013 1.0
[1371923210] Sat Jun 22 20:46:50 2013 3.0
[1371923270] Sat Jun 22 20:47:50 2013 4.0
[1371923330] Sat Jun 22 20:48:50 2013 6.0
[1371923390] Sat Jun 22 20:49:50 2013 7.0
[1371923450] Sat Jun 22 20:50:50 2013 9.0
[1371923510] Sat Jun 22 20:51:50 2013 10.0
[1371923570] Sat Jun 22 20:52:50 2013 nan
[1371923630] Sat Jun 22 20:53:50 2013 nan
[1371923690] Sat Jun 22 20:54:50 2013 nan
[1371923750] Sat Jun 22 20:55:50 2013 nan
[1371923810] Sat Jun 22 20:56:50 2013 nan
[1371923870] Sat Jun 22 20:57:50 2013 nan
[1371923930] Sat Jun 22 20:58:50 2013 nan
[1371923990] Sat Jun 22 20:59:50 2013 nan
[1371924050] Sat Jun 22 21:00:50 2013 nan
[1371924110] Sat Jun 22 21:01:50 2013 nan
[1371924170] Sat Jun 22 21:02:50 2013 11.0

Each datapoint value now describes 60sec interval instead of 10sec.
We can write a simple script that will read a number of @10.slice files (for some defined period), aggregate values (calculate 60sec Avg, or Max or Min) and write to new @60.slice file. For example, this is exactly what ceres-maintenance rollup plugin does.

Unfortunately,
ceres-node-read does not respect individual slice timeStep values, it takes the timeStemp from '.ceres-node' file, while the slicecat takes it from slice file name directly ('@60.slice').

If there are many slides with different 'timeStep' (for example 1371922150@60.slice and 1371923150@10.slice), ceres-node-read will read them with the same 'timeStep' value taken from .ceres-node file, and obviously, result will be incorrect.