Milestone: 1
dump_hist command, from a specified directory. It reformats, sorts, pivots (from skinny to wide) the data and produces a stream of events suitable for feeding to other Logstash operators, most typically scacsv.
input {
scabmcfile {
path => ... # string (required)
done_dir => ... # string (required)
ready_file => ... # string (optional), default: ""
tz_offset => ... # number (optional), default: 0
groups => ... # array (optional), []: 0
poll_interval => ... # number (optional), default: 10
}
}
input. Taking a look at a typical input file will help understand those.
server002/MEMORY.MEMORY/MEMFreeMem
Thu Nov 6 12:40:00 2014 137356
Thu Nov 6 12:41:00 2014 138552
...
Thu Nov 6 12:45:00 2014 132500
server002/CPU.CPU/CPUCpuUtil
Thu Nov 6 13:23:00 2014 58.7869
Thu Nov 6 13:24:00 2014 57.3054
...
Thu Nov 6 13:31:00 2014 53.9605
server002/MEMORY.MEMORY/MEMUsedMem
Thu Nov 6 12:40:00 2014 237356
Thu Nov 6 12:41:00 2014 238552
...
Thu Nov 6 12:44:00 2014 232472
server002/MEMORY.MEMORY/MEMFreeMem
Thu Nov 6 12:45:00 2014 105888
Thu Nov 6 12:46:00 2014 132940
...
Thu Nov 6 12:51:00 2014 166220
Total matched parameters: 4
First, there is key Node(or server,host etc), Group, Instance and Metric identity information at the start of each section. This is followed by actual timestamp and metric values (indented). The basic pattern is node/group.instance/metric and these fields are used to form the appropriate identities. This information is prepended to the subsequent metric lines to form a set of events. Then this set of events are sorted by time. This is required because the data in the file is ordered by resource etc. info rather than time. Finally, the events are pivoted from the essentially skinny format, to the required wide.
The output is a stream of events which can be passed directly to a scacsv output.
- Value type is String
- There is no default for this setting
Path to a directory where input files are to be found . File glob/wildcards are acceptable.
- Value type is String
- There is no default for this setting
When each file from the input path is processed, it will be moved to the directory specified by done_dir.
Note. It is the user responsibility to clean up the content in this directory e.g. by establish periodic jobs to delete content
- Value type is String
- The default value is ""
If this string has value, that value should be the full pathname to a file, which will be used as a token file. When scabmcfile find this token file, it will delete it, and then process the files in the directory in the normal way. If the string is empty, then files are processed normally. The presence or absence of such a token file can be used to control when file occurs. For example, you can keep moving files into the directory, but they will only be picked up if the token file is present.
- Value type is Number
- The default value is 0
A number of seconds to add to the epoch time of each event. Use this to ensure that the starttime and endtime assigned downstream by scabmcfile will be properly aligned. This is sometimes necessary as the plugin converts the timestamp in the data to epoch time, and the offset applied depends on the timezone of the host system, so you can use this setting to compensate.
- Value type is Array
- The default value is [], meaning 'allow all groups'
A set of group names, which specify which groups of metrics are to be processed. If the array is empty (default), then all groups encountered will be processed. If one or more group names are provided, then only those will be processed. This is extremely useful in filtering larger sets of data, where we want to remove some groups. Ideally, only data we wish to process would be exported, but this is not always the way
- Value type is Number
- The default value is 10
The number of seconds to wait between reading filenames from the directory.
Hello! I'm your friendly footer. If you're actually reading this, I'm impressed.