Using Magic Commands in Azure Synapse Notebooks | Examples & Tips
๐ What Are Magic Commands?
In Azure Synapse Notebooks, magic commands allow you to switch the execution context or language in a specific cell. These commands start with %%
and tell Synapse which language or runtime to use in the cell.
๐งช Common Magic Commands in Synapse
%%pyspark
– Run Python code using PySpark%%spark
– Run Scala code%%sql
– Execute Spark SQL queries%%csharp
– Run .NET for Apache Spark code%%md
– Write Markdown text (headings, notes)
✅ Example: PySpark Cell
%%pyspark
df = spark.read.csv('abfss://data@account.dfs.core.windows.net/sample.csv', header=True)
df.show()
✅ Example: Spark SQL Cell
%%sql
SELECT TOP 10 * FROM mytable
✅ Example: Markdown Cell
%%md
### This is a Markdown cell
Use this for annotations and titles
⚙️ How to Use Magic Commands
- Click + Code in Synapse Notebook
- Begin the cell with the desired magic command (e.g.,
%%pyspark
) - Write your code below and run the cell
๐ก Tips
- Use
%%sql
for querying Delta/Parquet data from Spark tables - Use
%%md
to format notebook sections and add comments - Stick to one language per cell; switching context within a cell is not supported
๐ Best Practices
- Label each section of your notebook with
%%md
to keep it organized - Run small chunks of code in separate cells for easier debugging
- Use PySpark for transformation logic and SQL for querying
๐บ Watch the Video Tutorial
๐ Credit: Content created with the help of ChatGPT and Gemini.