Executing or calling an Azure Databricks Notebook within ADF pipeline is quite easy. There is an in-built activity called “Notebook” available there under the Databricks
Category: databricks
Spark and Databricks Runtime Version
You can check the version of Spark running on the cluster your notebook is attached to as follows – … and to check the Databricks
Default Apache Spark Variables in Azure Databricks Notebook
I came across some python code in one of my notebooks and realized that a variable called “sc” was referenced there which was not defined
How to Get Path of Current Notebook
In an earlier post, we saw how to get the absolute path of a notebook in Azure Databricks by navigating GUI in the workspace. In
Python Cheatsheet
Something that I came across today, a useful consolidated view of python syntaxes so realized to share with everyone visiting Kloudspro. Could prove a handy
Azure Databricks Connection to SQL Server Using JDBC
In this post, we are going to see how to establish a connection to Azure SQL Server Database from within Azure Databricks Notebooks. We will
How to Fetch Azure Key Vault Secrets in Azure Databricks
Assuming that you have created Azure Key Vault scope in Azure Databricks and created some secrets in the vault, you can use the below commands
Installing/Setting Up Databricks CLI
Databricks CLI is a command line interface to Databricks platform. This is built on top of databricks REST APIs. Its commands are organized into different
How to Run a Databricks Notebook from Another Notebook
You can use %run command to run another notebook in your current notebook. In fact, it includes or concatenates another notebook in your notebook. This
DBFS
DBFS stands for Databricks File System. It provides a place to put your data in databricks. It is a distributed file system mounted into Azure
Parquet File Format
When it comes to storing data in files, CSV has been the most common format so far or may be delimited-file format, in a general
How to Mount Azure Blob Storage Container to Azure Databricks File System
In the previous post How to Configure Azure KeyVault in Azure Databricks, we saw how your Azure Databrick’s workspace could be linked to an Azure
How to Configure Azure KeyVault in Azure Databricks
Below are the things you would need to keep handy while configuring the Azure keyvault in Azure Databricks – Azure Databricks Host: URL in the
How to Check Python Version Installed on the Machine
Probably, the smallest post 🙂 Run the following command from command line prompt
Eager and Lazy Execution
Eager execution means instant executions as soon as the thread assigned to statement gets its turn and Lazy execution means, actual execution is delayed until
