Welcome To TechBrothersIT

Using Magic Commands in Azure Synapse Notebooks | Examples & Tips

📘 What Are Magic Commands?

In Azure Synapse Notebooks, magic commands allow you to switch the execution context or language in a specific cell. These commands start with %% and tell Synapse which language or runtime to use in the cell.

🧪 Common Magic Commands in Synapse

%%pyspark – Run Python code using PySpark
%%spark – Run Scala code
%%sql – Execute Spark SQL queries
%%csharp – Run .NET for Apache Spark code
%%md – Write Markdown text (headings, notes)

✅ Example: PySpark Cell

%%pyspark
df = spark.read.csv('abfss://data@account.dfs.core.windows.net/sample.csv', header=True)
df.show()

✅ Example: Spark SQL Cell

%%sql
SELECT TOP 10 * FROM mytable

✅ Example: Markdown Cell

%%md
### This is a Markdown cell
Use this for annotations and titles

⚙️ How to Use Magic Commands

Click + Code in Synapse Notebook
Begin the cell with the desired magic command (e.g., %%pyspark)
Write your code below and run the cell

💡 Tips

Use %%sql for querying Delta/Parquet data from Spark tables
Use %%md to format notebook sections and add comments
Stick to one language per cell; switching context within a cell is not supported

📈 Best Practices

Label each section of your notebook with %%md to keep it organized
Run small chunks of code in separate cells for easier debugging
Use PySpark for transformation logic and SQL for querying

📺 Watch the Video Tutorial

📚 Credit: Content created with the help of ChatGPT and Gemini.