This is the third part of a five-part series on automating common DevOps tasks in New Relic.
The first part shows you how to audit your services with Python and the NerdGraph API, and also includes the sample API code that this post uses. The second part shows you how to automate your tags with Python and the NerdGraph API.
All of the code from this five-part series can be found in this repository.
There are many reasons you might want to export your telemetry data—for instance, you might use a self-managed data platform or leverage a cloud platform as a service (PaaS) to do additional data analysis. Fortunately, you can use the NerdGraph API, a GraphQL-format API that lets you query New Relic data, to export your telemetry data.
This post shows you how to export telemetry data using New Relic Query Language (NRQL) and Python. This example uses The Four Golden Signals, but you can also apply this example to your own custom queries as well.
Prerequisites
If you plan to follow along with this tutorial and use Python to make your API calls, you need:
- Python 3 installed on your machine. Download and install it from python.org. Note that you can also make API calls to the specified endpoints using languages other than Python as well.
- With Python, you’ll be using the requests and json modules.
- A New Relic account. Sign up for free.
- An application instrumented with New Relic APM and infrastructure. Follow these installation steps to instrument an application.
- A New Relic API key. Follow these steps to get an API key, or, if you’re already signed in, generate an API key on the API keys page. To learn more about querying and configuration, see New Relic API keys.
- The Python starter code, which is in part one of this blog series.
The next video shows you how to export telemetry data using the NerdGraph API.
Exporting data for analysis
Here's an example query for exporting data using the NerdGraph API. If you need more context on the starter code, see Using Python to make an API call in part one of this series.
import requests
import json
from license import user_key
def nerdgraph_nrql(key):
# GraphQL query to NerdGraph
query = """
{
actor { account(id: XXX)
{ nrql
(query: "XXX")
{ results } } }
}"""
# NerdGraph endpoint
endpoint = "https://api.newrelic.com/graphql"
headers = {'API-Key': f'{key}'}
response = requests.post(endpoint, headers=headers, json={"query": query})
if response.status_code == 200:
# convert a JSON into an equivalent python dictionary
dict_response = json.loads(response.content)
print(dict_response['data']['actor']['account']['nrql'])
# optional - serialize object as a JSON formatted stream
# json_response = json.dumps(response.json(), indent=2)
# print(json_response)
else:
# raise an exepction with a HTTP response code, if there is an error
raise Exception(f'Nerdgraph query failed with a {response.status_code}.')
nerdgraph_nrql(user_key)Note that you would replace the "XXX" in (query: "XXX") with the data you'd like to export.
Using NRQL to query data on golden signals
So what kind of data would you potentially export? Here's where the golden signals come in. The Four Golden Signals of monitoring are latency, traffic, errors, and saturation. If you can only measure four metrics of your user-facing system, focus on these signals.
If you're new to NRQL, see this resource on NRQL syntax, clauses, components, and functions.
Use FACET to separate and group your results by attribute values. For example, you could FACET your PageView data by deviceType to figure out what percentage of your traffic comes from mobile, tablet, and desktop devices.
Use the LIMIT clause to specify how many facets appear (the default is 10). For more complex grouping, use FACET CASES. These clauses support up to five attributes, separated by commas.
Querying latency
The next two queries return the following:
- The average response time since 7 days ago, grouped hourly
- The 95th percentile response time since 30 days ago, grouped daily
You'll need to specify your appName and the httpResponseCode you are searching for—for instance, you could use 200 for successful page views.
SELECT average(duration) FROM Transaction WHERE appName = 'XXX' AND httpResponseCode = 'XXX' SINCE 7 days ago limit MAX FACET hourOf(timestamp)
SELECT percentile(duration, 95) FROM Transaction WHERE appName = 'XXX' AND httpResponseCode = 'XXX' SINCE 30 days ago limit MAX FACET dateOf(timestamp)Querying traffic
The next two queries return the following:
- The rate of change (1min) since 7 days ago, grouped hourly
- Return the rate of change (1hr) since 30 days ago, grouped daily
Once again, you need to specify your appName.
SELECT rate(count(*), 1 minute) FROM Transaction WHERE appName = 'XXX' SINCE 7 days ago limit MAX FACET hourOf(timestamp)
SELECT rate(count(*), 1 hour) FROM Transaction WHERE appName = 'XXX' SINCE 30 days ago limit MAX FACET dateOf(timestamp)Querying errors
The next two queries return the following:
- The percentage of errors for status code 2XX since 7 days ago, grouped hourly
- The percentage of errors for status codes outside 2XX since 30 days ago, grouped daily
SELECT percentage(count(*), WHERE httpResponseCode LIKE '2%') FROM Transaction WHERE appName = 'XXX' SINCE 7 days ago limit MAX FACET hourOf(timestamp)
SELECT percentage(count(*), WHERE httpResponseCode NOT LIKE '2%') FROM Transaction WHERE appName = 'XXX' SINCE 30 days ago limit MAX FACET dateOf(timestamp)Querying saturation
The next two queries return the following:
- The average CPU since 7 days ago, grouped hourly
- The average MEM since 30 days ago, grouped daily
SELECT average(cpuPercent) FROM SystemSample WHERE hostname LIKE '%XXX%' SINCE 7 days ago limit MAX FACET hourOf(timestamp)
SELECT average(memoryUsedPercent) FROM SystemSample WHERE hostname LIKE '%XXX%' SINCE 30 days ago limit MAX FACET dateOf(timestamp)Stay tuned for part four in the series, where you'll learn how to export dashboards for reporting with Python and the NerdGraph API. In the mean time, learn about our python agent for event loop diagnostics.
Next steps
Learn what else you can do with the Nerdgraph API and continue to build DevOps into your observability practice. If you don't have a New Relic account yet, sign up today. Your free account includes 100 GB/month of free data ingest, one free full-access user, and unlimited free basic users.
Read the next part in this 5-part series: Export dashboards with Python and the NerdGraph API.
Want to learn more about how New Relic partners with customers for success? Read up on our collaboration with ZenHub.
The views expressed on this blog are those of the author and do not necessarily reflect the views of New Relic. Any solutions offered by the author are environment-specific and not part of the commercial solutions or support offered by New Relic. Please join us exclusively at the Explorers Hub (discuss.newrelic.com) for questions and support related to this blog post. This blog may contain links to content on third-party sites. By providing such links, New Relic does not adopt, guarantee, approve or endorse the information, views or products available on such sites.
