Metrics introduction
Prometheus is an open-source systems monitoring and alerting toolkit originally built at SoundCloud.
Taurus is using Prometheus for collecting and storing its metrics as time series data, i.e. metrics information is stored with the timestamp at which it was recorded, alongside optional key-value pairs called labels.
Metrics are numerical measurements. The term time series refers to the recording of changes over time. Metrics play an important role in understanding why your application is working in a certain way.
Metrics Family and Description
Those metrics can be accessed through a call to:
https://explorers_url/metrics
GO metrics
| Metric Name | Metric Description |
|---|---|
| go_gc_duration_seconds | A summary of the wall-time pause (stop-the-world) duration in garbage collection cycles. |
| go_gc_gogc_percent | Heap size target percentage configured by the user, otherwise 100. This value is set by the GOGC environment variable, and the runtime/debug.SetGCPercent function. Sourced from /gc/gogc:percent. |
| go_gc_gomemlimit_bytes | Go runtime memory limit configured by the user, otherwise math.MaxInt64. This value is set by the GOMEMLIMIT environment variable, and the runtime/debug.SetMemoryLimit function. Sourced from /gc/gomemlimit:bytes. |
| go_goroutines | Number of goroutines that currently exist. |
| go_info | Information about the Go environment. |
| go_memstats_alloc_bytes | Number of bytes allocated in heap and currently in use. Equals to /memory/classes/heap/objects:bytes. |
| go_memstats_alloc_bytes_total | Total number of bytes allocated in heap until now, even if released already. Equals to /gc/heap/allocs:bytes. |
| go_memstats_buck_hash_sys_bytes | Number of bytes used by the profiling bucket hash table. Equals to /memory/classes/profiling/buckets:bytes. |
| go_memstats_frees_total | Total number of heap objects frees. Equals to /gc/heap/frees:objects + /gc/heap/tiny/allocs:objects. |
| go_memstats_gc_sys_bytes | Number of bytes used for garbage collection system metadata. Equals to /memory/classes/metadata/other:bytes. |
| go_memstats_heap_alloc_bytes | Number of heap bytes allocated and currently in use, same as go_memstats_alloc_bytes. Equals to /memory/classes/heap/objects:bytes. |
| go_memstats_heap_idle_bytes | Number of heap bytes waiting to be used. Equals to /memory/classes/heap/released:bytes + /memory/classes/heap/free:bytes. |
| go_memstats_heap_inuse_bytes | Number of heap bytes that are in use. Equals to /memory/classes/heap/objects:bytes + /memory/classes/heap/unused:bytes. |
| go_memstats_heap_objects | Number of currently allocated objects. Equals to /gc/heap/objects:objects. |
| go_memstats_heap_released_bytes | Number of heap bytes released to OS. Equals to /memory/classes/heap/released:bytes. |
| go_memstats_heap_sys_bytes | Number of heap bytes obtained from system. Equals to /memory/classes/heap/objects:bytes + /memory/classes/heap/unused:bytes + /memory/classes/heap/released:bytes + /memory/classes/heap/free:bytes. |
| go_memstats_last_gc_time_seconds | Number of seconds since 1970 of last garbage collection. |
| go_memstats_mallocs_total | Total number of heap objects allocated, both live and gc-ed. Semantically a counter version for go_memstats_heap_objects gauge. Equals to /gc/heap/allocs:objects + /gc/heap/tiny/allocs:objects. |
| go_memstats_mcache_inuse_bytes | Number of bytes in use by mcache structures. Equals to /memory/classes/metadata/mcache/inuse:bytes. |
| go_memstats_mcache_sys_bytes | Number of bytes used for mcache structures obtained from system. Equals to /memory/classes/metadata/mcache/inuse:bytes + /memory/classes/metadata/mcache/free:bytes. |
| go_memstats_mspan_inuse_bytes | Number of bytes in use by mspan structures. Equals to /memory/classes/metadata/mspan/inuse:bytes. |
| go_memstats_mspan_sys_bytes | Number of bytes used for mspan structures obtained from system. Equals to /memory/classes/metadata/mspan/inuse:bytes + /memory/classes/metadata/mspan/free:bytes. |
| go_memstats_next_gc_bytes | Number of heap bytes when next garbage collection will take place. Equals to /gc/heap/goal:bytes. |
| go_memstats_other_sys_bytes | Number of bytes used for other system allocations. Equals to /memory/classes/other:bytes. |
| go_memstats_stack_inuse_bytes | Number of bytes obtained from system for stack allocator in non-CGO environments. Equals to /memory/classes/heap/stacks:bytes. |
| go_memstats_stack_sys_bytes | Number of bytes obtained from system for stack allocator. Equals to /memory/classes/heap/stacks:bytes + /memory/classes/os-stacks:bytes. |
| go_memstats_sys_bytes | Number of bytes obtained from system. Equals to /memory/classes/total:byte. |
| go_sched_gomaxprocs_threads | The current runtime.GOMAXPROCS setting, or the number of operating system threads that can execute user-level Go code simultaneously. Sourced from /sched/gomaxprocs:threads. |
| go_threads | Number of OS threads created. |
Process metrics
| Metric Name | Metric Description |
|---|---|
| http_server_handling_seconds | Total duration of requests in seconds. |
| process_cpu_seconds_total | Total user and system CPU time spent in seconds. |
| process_max_fds | Maximum number of open file descriptors. |
| process_network_receive_bytes_total | Number of bytes received by the process over the network. |
| process_network_transmit_bytes_total | Number of bytes sent by the process over the network. |
| process_open_fds | Number of open file descriptors. |
| process_resident_memory_bytes | Resident memory size in bytes. |
| process_start_time_seconds | Start time of the process since unix epoch in seconds. |
| process_virtual_memory_bytes | Virtual memory size in bytes. |
| process_virtual_memory_max_bytes | Maximum amount of virtual memory available in bytes. |
| promhttp_metric_handler_requests_in_flight | Current number of scrapes being served. |
| promhttp_metric_handler_requests_total | Total number of scrapes by HTTP status code. |
Taurus blocksd metrics
| Metric Name | Metric Description |
|---|---|
| tg_blocksd_adresses_registered | The number of addresses followed. |
| tg_blocksd_block_indexed | Last block indexed. |
| tg_blocksd_blocks_fetched_total | Total blocks fetched. |
| tg_blocksd_event_inserted_total | Total events inserted. |
| tg_blocksd_health_check_failed | The number of failed health checks. |
| tg_blocksd_job_duration_seconds | Total duration of a job in seconds. |
| tg_blocksd_state_block_indexed | Collection's state last block indexed. |
| tg_blocksd_transition_block_indexed | Collection's transition last block indexed. |
| tg_blocksd_tx_broadcasted_total | Total tx broadcasted. |
| tg_blocksd_tx_inserted_total | Total tx inserted. |
Metrics Family and Representation
| Metric Name | Metric Representation |
|---|---|
| go_gc_duration_seconds | summary |
| go_gc_gogc_percent | gauge |
| go_gc_gomemlimit_bytes | gauge |
| go_goroutines | gauge |
| go_info | gauge |
| go_memstats_alloc_bytes | gauge |
| go_memstats_alloc_bytes_total | counter |
| go_memstats_buck_hash_sys_bytes | gauge |
| go_memstats_frees_total | counter |
| go_memstats_gc_sys_bytes | gauge |
| go_memstats_heap_alloc_bytes | gauge |
| go_memstats_heap_idle_bytes | gauge |
| go_memstats_heap_inuse_bytes | gauge |
| go_memstats_heap_objects | gauge |
| go_memstats_heap_released_bytes | gauge |
| go_memstats_heap_sys_bytes | gauge |
| go_memstats_last_gc_time_seconds | gauge |
| go_memstats_mallocs_total | counter |
| go_memstats_mcache_inuse_bytes | gauge |
| go_memstats_mcache_sys_bytes | gauge |
| go_memstats_mspan_inuse_bytes | gauge |
| go_memstats_mspan_sys_bytes | gauge |
| go_memstats_next_gc_bytes | gauge |
| go_memstats_other_sys_bytes | gauge |
| go_memstats_stack_inuse_bytes | gauge |
| go_memstats_stack_sys_bytes | gauge |
| go_memstats_sys_bytes | gauge |
| go_sched_gomaxprocs_threads | gauge |
| go_threads | gauge |
| http_server_handling_seconds | summary |
| process_cpu_seconds_total | counter |
| process_max_fds | gauge |
| process_network_receive_bytes_total | counter |
| process_network_transmit_bytes_total | counter |
| process_open_fds | gauge |
| process_resident_memory_bytes | gauge |
| process_start_time_seconds | gauge |
| process_virtual_memory_bytes | gauge |
| process_virtual_memory_max_bytes | gauge |
| promhttp_metric_handler_requests_in_flight | gauge |
| promhttp_metric_handler_requests_total | counter |
| tg_blocksd_adresses_registered | gauge |
| tg_blocksd_block_indexed | gauge |
| tg_blocksd_blocks_fetched_total | counter |
| tg_blocksd_event_inserted_total | counter |
| tg_blocksd_health_check_failed | gauge |
| tg_blocksd_job_duration_seconds | summary |
| tg_blocksd_state_block_indexed | gauge |
| tg_blocksd_transition_block_indexed | gauge |
| tg_blocksd_tx_broadcasted_total | counter |
| tg_blocksd_tx_inserted_total | counter |