Using Magic Commands in Azure Synapse Notebooks | Examples & Tips | #azuresynapse #pyspark | Azure Synapse Analytics Tutorial

Using Magic Commands in Azure Synapse Notebooks | Examples & Tips

Using Magic Commands in Azure Synapse Notebooks | Examples & Tips

๐Ÿ“˜ What Are Magic Commands?

In Azure Synapse Notebooks, magic commands allow you to switch the execution context or language in a specific cell. These commands start with %% and tell Synapse which language or runtime to use in the cell.

๐Ÿงช Common Magic Commands in Synapse

  • %%pyspark – Run Python code using PySpark
  • %%spark – Run Scala code
  • %%sql – Execute Spark SQL queries
  • %%csharp – Run .NET for Apache Spark code
  • %%md – Write Markdown text (headings, notes)

✅ Example: PySpark Cell

%%pyspark
df = spark.read.csv('abfss://data@account.dfs.core.windows.net/sample.csv', header=True)
df.show()

✅ Example: Spark SQL Cell

%%sql
SELECT TOP 10 * FROM mytable

✅ Example: Markdown Cell

%%md
### This is a Markdown cell
Use this for annotations and titles

⚙️ How to Use Magic Commands

  • Click + Code in Synapse Notebook
  • Begin the cell with the desired magic command (e.g., %%pyspark)
  • Write your code below and run the cell

๐Ÿ’ก Tips

  • Use %%sql for querying Delta/Parquet data from Spark tables
  • Use %%md to format notebook sections and add comments
  • Stick to one language per cell; switching context within a cell is not supported

๐Ÿ“ˆ Best Practices

  • Label each section of your notebook with %%md to keep it organized
  • Run small chunks of code in separate cells for easier debugging
  • Use PySpark for transformation logic and SQL for querying

๐Ÿ“บ Watch the Video Tutorial

๐Ÿ“š Credit: Content created with the help of ChatGPT and Gemini.