Once again I’ve been back into the Wavefront CLI and Ruby SDK, adding more functionality. This time, the work was driven by a wish to have our EC2 instances tag themselves with things like account, instance type, the repository from which they take their configuration management code, or maybe the version of that code.
The Ruby SDK didn’t give you any help in doing this, so the first job was to extend the class offering wrappers around the appropriate API calls.
Once that was done (including
of course!), I added a new
source command to the CLI. As we use
docopt to parse input, the hardest part of the job
was coming up with a sensible interface. After a lot of thought and a
couple of rewrites, I ended up with this.
$ wavefront source --help Usage: wavefront source list [-f format] [-at] [-T tag ...] [-s source] [-l limit] <pattern> wavefront source show [-f format] <host> ... wavefront source describe <description> [<host>] ... wavefront source undescribe [<host>] ... wavefront source tag add [-H host ... ] <tag> ... wavefront source tag delete [-H host ... ] <tag> ... wavefront source untag [<host>] ... Options: -a, --all including hidden sources in 'human' output -t, --tags show tag counts in 'human' output -s, --start=STRING start the list after the named source -l, --limit=NUMBER only list NUMBER sources -H, --host=STRING source to manipulate -f, --format=STRING output format (ruby, json, human) [default: human]
For the sake of clarity I’ve removed the global options from the output. These let you define your Wavefront endpoint, authentication token and so-on. (The beauty of docopt is that you only have to write the help message shown above as a block of text in your code, and it all those options and subcommands get magically populated.)
The general principle of the
source command is to have the things you
are setting (tags and descriptions) be arguments, and the things you are
setting them on (hosts, or, techinically, “sources”), options. This way
you can set multiple tags on multiple hosts with a single command.
If, when manipulating a tag or description, you don’t specify a source
(or sources), then the CLI will assume you wish to refer the machine on
which it is running. That way automated “tagging myself” jobs don’t need
to keep shelling out to
uname(1) or whatever.
Let’s do a few examples.
In my home lab I have a Solaris server called
shark. It has a number
of zones on it, whose names all begin
shark-. Let’s list everything
shark in the name.
$ wavefront source list shark HOSTNAME DESCRIPTION TAGS shark shark-media shark-wavefront shark-ws
shark, is a pattern, which is passed straight through
to Wavefront. It seems that there it is matched as a general
regex, because without having to put
.* or similar on the end of
it, I’ve got all the
shark zones for free. It’s not clear from
that what’s what, so let’s quickly describe them
$ wavefront source describe -H shark 'general-purpose server' setting 'shark' description to 'general-purpose server' $ wavefront source describe -H shark-media 'minidlna server' setting 'shark-media' description to 'minidlna server' $ wavefront source describe -H shark-wavefront 'wavefront proxy' setting 'shark-wavefront' description to 'wavefront proxy' $ wavefront source list shark HOSTNAME DESCRIPTION TAGS shark general-purpose server shark-media minidlna server shark-wavefront wavefront proxy shark-ws
That’s fine, but descriptions aren’t particularly useful. As far as I know you can’t do anything with them in the timeseries language. Let’s do some tags.
$ wavefront source tag add -H shark physical solaris Tagging 'shark' with 'physical' Tagging 'shark' with 'solaris' $ wavefront source tag add -H shark-media -H shark-wavefront -H shark-ws zone Tagging 'shark-media' with 'zone' Tagging 'shark-wavefront' with 'zone' Tagging 'shark-ws' with 'zone' $ wavefront source list shark HOSTNAME DESCRIPTION TAGS shark general-purpose server physical shark-media minidlna server zone shark-wavefront wavefront proxy zone shark-ws zone
Now, we can filter our
ts() queries. Putting
tag=physical to any
ts() statement (as the second argument, just before closing the
brackets) will only show data for the physical host. Zones will be
ignored. Sources can have as many tags as you want, so you can
create all manner of ways to cross-reference.
We incorporate our host’s product, enviroment, and role into our metric paths, but you might feel tagging is a better way to do that. Also tag by AZ, instance-type and it’s trivial to pull out JVM stats just for t2-medium Cassandra boxes in the eu-west-1 production environment.
You can take tags away just as you added them, by swapping
delete, or remove all a source’s tags in one go with
Similarly, there’s an
undescribe command, which doesn’t
technically remove a description (the API doesn’t provide a method
for that) but does set it to an empty string.
To see everything Wavefront knows about a particular source, ask the
show it to you.
$ wavefront source show shark shark description general-purpose server tags physical
list command, you are likely to have a lot of
sources. (We have over 150,000.) So, if you’re
listing, you likely
want to be careful with your pattern, or paginate. To help you do
list command provides
--limit, to limit the number of
sources it returns, and
--start. The latter takes requires a
source name, and when it is used, the command will return sources
starting with the one after that which you supplied.
Requesting a source list in
json format will include
hidden sources, but
human format will not, unless you supply the
-a option. The same goes for the tag count summary included in
list output: to turn that on in human-readable mode, add
$ wavefront source list -at shark HOSTNAME DESCRIPTION TAGS shark general-purpose server physical shark-media minidlna server zone shark-wavefront wavefront proxy zone shark-ws zone shark-wslab hidden TAG COUNT hidden 339 physical 1 zone 3
It’s also possible to have the human-readable output only display
sources which have a particular tag, or tags. To do this, use