02 Transformations

Transform and process data in Grafana.

What You'll Learn

• Understand what transformations do and when to use them

• Apply common transformations (Reduce, Organize, Filter, Rename)

• Use regression analysis for trend lines

• Combine data from multiple queries

• Chain transformations for complex data manipulation

Prerequisites

• Completed Part 1 and Part 2 tutorials

• Familiarity with basic panel creation

---

Overview

Transformations manipulate data after it's been queried but before it's visualized. They let you:

• Aggregate data – Calculate min, max, mean across time

• Reorganize data – Rename fields, reorder columns, hide data

• Combine data – Merge queries, join tables

• Analyse trends – Add regression lines, calculate differences

• Filter data – Show only specific fields or values

When to Use Transformations

Scenario	Transformation
Show summary statistics in a table	Reduce
Rename technical field names	Rename by regex / Organize fields
Add a trend line to time series	Regression
Combine data from two queries	Merge / Join
Show only specific variables	Filter by name

---

The Transform Tab

Accessing Transformations

1. Edit a panel

2. Click the Transform tab (between Query and Alert)

   [SCREENSHOT: Transform tab location in panel editor]

3. Click + Add transformation

4. Select a transformation from the list

   [SCREENSHOT: Transformation selection list]

Transformation Order

Transformations execute in order from top to bottom. You can:

• Drag transformations to reorder

• Disable a transformation (eye icon) to test

• Delete transformations (trash icon)

  [SCREENSHOT: Multiple transformations with drag handles]

Important: Order matters! A filter applied before Reduce gives different results than after.

---

Common Transformations

Reduce

Converts time series data into aggregated values – perfect for summary tables.

How it works:

• Takes all data points in each series

• Calculates selected statistics

• Outputs one row per series

Configuration:

1. Add Reduce transformation

2. Select calculations:

   Calculation	Description
   Last *	Most recent non-null value
   First *	First non-null value
   Min	Minimum value
   Max	Maximum value
   Mean	Average value
   Sum	Total of all values
   Count	Number of data points
   Range	Max - Min
   Difference	Last - First
   Difference percent	% change
   All values	Array of all values

   [SCREENSHOT: Reduce transformation options]

Example: Turn a time series into a summary table showing current value, min, max for each variable.

---

Organize Fields by Name

Rename, reorder, and hide columns – essential for clean tables.

How it works:

• Lists all fields in the data

• Lets you rename, reorder, or hide each field

Configuration:

1. Add Organize fields by name transformation

2. For each field:

   • Rename: Enter new display name

   • Hide: Click eye icon to hide

   • Reorder: Drag to change position

   [SCREENSHOT: Organize fields transformation with renamed columns]

Example: Rename "92694f02-a0c7-4900-8758-c9a2eb4324f0" to "Throughput".

---

Rename by Regex

Rename multiple fields at once using pattern matching.

How it works:

• Matches field names against a regex pattern

• Replaces with a new name (can use capture groups)

Configuration:

Setting	Description
Match	Regex pattern to match
Replace	Replacement text (use $1, $2 for groups)

Examples:

Match	Replace	Before	After
(.*)	$1 (kg/h)	Durchsatz	Durchsatz (kg/h)
Extruder (.) Durchsatz	Ext $1	Extruder A Durchsatz	Ext A
\(Soll\)	(Target)	Speed (Soll)	Speed (Target)

[SCREENSHOT: Rename by regex transformation]

---

Filter by Name

Show only specific fields in the output.

How it works:

• Select which fields to include or exclude

• Other fields are hidden from visualization

Configuration:

1. Add Filter by name transformation

2. Choose mode:

   • Include: Only show selected fields

   • Exclude: Hide selected fields

3. Select fields using checkboxes or regex

   [SCREENSHOT: Filter by name transformation]

Example: Show only temperature fields from a query that returns many variables.

---

Filter Data by Values

Filter rows based on field values.

How it works:

• Evaluates each row against conditions

• Removes rows that don't match

Configuration:

Setting	Description
Filter type	Include / Exclude
Condition	Match all / Match any
Field	Which field to check
Match	Comparison type (Greater, Less, Equal, etc.)
Value	Comparison value

[SCREENSHOT: Filter data by values transformation]

Example: Show only data points where temperature > 200°C.

---

Regression Analysis

Add trend lines to understand data direction over time.

Adding a Regression Line

1. Create a time series panel with your data

2. Add Regression analysis transformation

3. Configure:

   Setting	Description
   X field	Time field (usually "time")
   Y field	Value field to analyse

[SCREENSHOT: Regression transformation configuration]

1. A new series appears showing the linear trend line

2. Add override to place the regression line on the same axis as original data. The regression line has no unit and will be placed on a different y-axis by default. This can create wrong visual impressions. By setting the unit of the regression to the same unit as the input variable they will be displayed on the same axis.

Interpreting Results

The regression line shows the overall trend:

• Upward slope: Values increasing over time

• Downward slope: Values decreasing over time

• Flat line: Values stable

[SCREENSHOT: Time series with original data and regression line]

Styling the Trend Line

Use overrides to style the regression line differently:

1. Add override for the regression series (often named same as original + suffix)

   • Fields with name matching regex .*predicted.*

2. Set unit to match original data unit (to place on same axis)

3. Set different color (e.g., red for trend)

4. Set dashed line style

5. Reduce opacity

---

Combining Data

Merge data from multiple queries or sources.

Merge

Combines all series into a single table.

How it works:

• Takes all fields from all queries

• Creates one table with all columns

• Aligns rows by time (for time series)

Configuration:

1. Add Merge transformation

2. No additional settings needed

[SCREENSHOT: Merge transformation combining two queries]

Use case: Combine data from two machines into one comparison table.

---

Join by Field

Combines tables that share a common field.

How it works:

• Matches rows where the join field has the same value

• Creates new table with columns from both

Configuration:

Setting	Description
Field	Which field to join on
Mode	Inner, Left outer, Right outer

Join modes:

Mode	Result
Inner	Only rows matching in both tables
Left outer	All from first table, matching from second
Right outer	All from second table, matching from first

[SCREENSHOT: Join by field transformation]

Use case: Join machine data with product data using production run ID.

---

Concatenate Fields

Combines multiple series into one.

How it works:

• Stacks data from multiple series vertically

• Creates one unified series

Configuration:

1. Add Concatenate fields transformation

2. Select the frame name for the result

Use case: Combine three separate throughput queries into one combined series.

---

Transformation Chains

Complex data manipulation often requires multiple transformations in sequence.

Example: Summary Table with Trend Indicators

Goal: Show a table with variable names, current values, min/max, and trend sparklines.

Transformations (in order):

1. Reduce

   • Calculations: Last *, Min, Max, All values

2. Organize fields by name

   • Rename "Field" → "Variable"

   • Rename "All values" → "Trend"

   • Reorder columns

3. Filter by name (if needed)

   • Hide intermediate fields

[SCREENSHOT: Multiple transformations creating summary table]

Example: Filtered Comparison

Goal: Compare throughput from two extruders, showing only when both exceed 50 kg/h.

Transformations (in order):

1. Merge (combine both queries)

2. Filter data by values

   • Extruder A Durchsatz > 50

   • AND Extruder B Durchsatz > 50

---

Practical Examples

Example 1: Variable Summary Dashboard

Create a table showing all machine variables with statistics.

Setup:

1. Query multiple variables from a machine

2. Set visualization to Table

Transformations:

Step	Transformation	Configuration
1	Reduce	Last *, Min, Max, Mean, All values
2	Organize fields	Rename Field→Variable, All values→Sparkline

Result: Clean summary table with sparklines.

[SCREENSHOT: Final summary table panel]

---

Example 2: Trend Analysis

Show throughput with a trend line to identify production patterns.

Setup:

1. Query throughput variable

2. Set visualization to Time Series

Transformations:

Step	Transformation	Configuration
1	Regression analysis	X: time, Y: throughput field

Styling:

• Override regression series with red, dashed line

[SCREENSHOT: Time series with regression trend line]

---

Example 3: Multi-Machine Comparison Table

Compare the same metric across multiple machines.

Setup:

1. Query A: Machine 1 throughput

2. Query B: Machine 2 throughput

3. Set visualization to Table

Transformations:

Step	Transformation	Configuration
1	Reduce	Last *
2	Merge	(combines both queries)
3	Organize fields	Rename to friendly names

[SCREENSHOT: Multi-machine comparison table]

---

Tips and Best Practices

Performance Considerations

• Transformations run in the browser

• Large datasets can slow down rendering

• Use query-level filtering when possible (more efficient)

Debugging Transformations

1. Add transformations one at a time

2. Check the Table view toggle to see intermediate results

3. Use the Debug transformation to inspect data structure

Common Patterns

Pattern	Transformations
Time series → Summary table	Reduce → Organize fields
Add sparklines	Reduce (with All values) → Override cell type
Trend line	Regression analysis
Clean field names	Rename by regex or Organize fields
Compare machines	Merge → Organize fields

---

Summary

You've learned how to manipulate data with transformations:

• Aggregate data with Reduce

• Organize and rename fields

• Filter data by name or value

• Add regression trend lines

• Combine data from multiple queries

• Chain transformations for complex operations

---

What's Next?

You've completed Part 3: Intermediate Features. Continue with:

• Introduction to the ENLYZE API – API-based dashboards for advanced use cases

---

Quick Reference

Common Transformations

Transformation	Purpose
Reduce	Aggregate time series to statistics
Organize fields	Rename, reorder, hide columns
Rename by regex	Pattern-based renaming
Filter by name	Show/hide specific fields
Filter data by values	Remove rows by condition
Regression analysis	Add trend line
Merge	Combine queries horizontally
Join by field	Join tables on common field
Concatenate	Stack data vertically

Reduce Calculations

Calculation	Description
Last *	Most recent non-null
First *	First non-null
Min	Minimum
Max	Maximum
Mean	Average
Sum	Total
Count	Number of points
Range	Max - Min
Difference	Last - First
Difference percent	% change
All values	Array (for sparklines)

Transformation Order Tips

Goal	Order
Filter then aggregate	Filter → Reduce
Aggregate then filter	Reduce → Filter
Rename for display	Last transformation
Debug issues	Add Debug transformation