Tutorial #1 wrote a simple script to query the conn_stats
table of data provided by Pixie's platform:
1# Import Pixie's module for querying data2import px34# Load the last 30 seconds of Pixie's `conn_stats` table into a Dataframe.5df = px.DataFrame(table='conn_stats', start_time='-30s')67# Display the DataFrame with table formatting8px.display(df)
This tutorial will modify that script to produce a table that summarizes the total amount of traffic coming in and out of each of the pods in your cluster.
The ctx
function provides extra Kubernetes metadata context based on the existing information in your DataFrame.
Because the conn_stats
table contains the upid
(an opaque numeric ID that globally identifies a process running inside the cluster), PxL can infer the namespace
, service
, pod
, container
and cmd
(command) that initiated the connection.
We'll add columns for pod
and service
to our script.
1# Import Pixie's module for querying data2import px34# Load the last 30 seconds of Pixie's `conn_stats` table into a Dataframe.5df = px.DataFrame(table='conn_stats', start_time='-30s')67# Each record contains contextual information that can be accessed by the reading ctx.8df.pod = df.ctx['pod']9df.service = df.ctx['service']1011# Display the DataFrame with table formatting12px.display(df)
px live -f my_first_script.pxl
Use your arrow keys to scroll to the far right of the table and you should see a new columns labeled pod
and service
, representing the kubernetes entity that initiated the traced connection. Note that some of the connections in the table are missing context (a pod
or service
). This occasionally occurs due to a gap in metadata or a short-lived upid
.
Let's group the connection data by unique pairs of values in the pod
and service
columns, computing the aggregating expressions on each group of data.
Note that PxL does not currently support standalone groupings, you must always follow the groupby()
call with a call to agg()
. However, the agg call can take zero arguments. A full list of the aggregating functions is available here.
1# Import Pixie's module for querying data2import px34# Load the last 30 seconds of Pixie's `conn_stats` table into a Dataframe.5df = px.DataFrame(table='conn_stats', start_time='-30s')67# Each record contains contextual information that can be accessed by the reading ctx.8df.pod = df.ctx['pod']9df.service = df.ctx['service']1011# Group data by unique values in the 'pod' column and calculate the12# sum of the 'bytes_sent' and 'bytes_recv' for each unique pod grouping.13df = df.groupby(['pod', 'service']).agg(14 bytes_sent=('bytes_sent', px.sum),15 bytes_recv=('bytes_recv', px.sum)16)1718# Force ordering of the columns (do not include _clusterID_, which is a product of the CLI and not the PxL script)19df = df[['service', 'pod', 'bytes_sent', 'bytes_recv']]2021# Display the DataFrame with table formatting22px.display(df)
ctrl+c
and re-run the script. Each row in the output represents a unique pod
and service
pair that had one or more connections traced in the last 30 seconds. All of the connections between these pod
/ service
pairs have had their sent- and received- bytes summed for the 30 second time period.
Let's filter out the rows in the DataFrame that do not have a service identified (an empty value for the service
column).
1# Import Pixie's module for querying data2import px34# Load the last 30 seconds of Pixie's `conn_stats` table into a Dataframe.5df = px.DataFrame(table='conn_stats', start_time='-30s')67# Each record contains contextual information that can be accessed by the reading ctx.8df.pod = df.ctx['pod']9df.service = df.ctx['service']1011# Group data by unique values in the 'pod' column and calculate the12# sum of the 'bytes_sent' and 'bytes_recv' for each unique pod grouping.13df = df.groupby(['pod', 'service']).agg(14 bytes_sent=('bytes_sent', px.sum),15 bytes_recv=('bytes_recv', px.sum)16)1718# Force ordering of the columns (do not include _clusterID_, which is a product of the CLI and not the PxL script)19df = df[['service', 'pod', 'bytes_sent', 'bytes_recv']]2021# Filter out connections that don't have their service identified.22df = df[df.service != '']2324# Display the DataFrame with table formatting25px.display(df)
ctrl+c
and re-run the script. The script output shows that the rows that were missing a service
value are no longer included in the table.
You now have a script that produces a table summarizing the total amount of traffic coming in and out of each of the pods in your cluster for the last 30s (start_time
). This could be used to:
pod
's incoming vs outgoing traffic.pods
under the same service
receive a similar amount of traffic or if there is an imbalance in traffic received.