databricks connect to SQL db

Publié le: 21 mars 2022
sur la chaîne: Knowledge Sharing

25,492

115

this video provides information on how to connect azure SQL db from datbaricks and transfer the data

here is the code

df = spark.read.csv('dbfs:/FileStore/tables/Legally_Operating_Businesses.csv',header="true")
display(df)

%scala
val schema = "DCALicense string,LicenseType string,LicenseExpirationDate string,LicenseStatus string,LicenseCreationDate string,Industry string,BusinessName string,BusinessName2 string,AddressBuilding string,AddressStreetName string,SecondaryAddressStreetName string,AddressCity string,AddressState string,AddressZIP string,ContactPhoneNumber string,AddressBorough string,BoroughCode string,CommunityBoard string,CouncilDistrict string,BIN string,BBL string,NTA string,CensusTract string,Detail string,Longitude string,Latitude string,Location string"

%scala

import java.util.Properties

val jdbcHostName = "dbricksconnectivity.database.windows.net"
val jdbcPort =1433
val jdbcdbName = "dbricksCon"
val myproperties = new Properties()

val jdbcUsername = dbutils.secrets.get(scope = "bricksScope", key = "sqlusername")
val jdbcPassword = dbutils.secrets.get(scope = "bricksScope", key = "sqlpassword")

myproperties.put("user",jdbcUsername)
myproperties.put("password",jdbcPassword)

val url = s"jdbc:sqlserver://${jdbcHostName}:${jdbcPort};database=${jdbcdbName}"
val driverClass = "com.microsoft.sqlserver.jdbc.SQLServerDriver"
myproperties.setProperty("Driver", driverClass)

val mydf = spark.read.format("csv").option("header","true").schema(schema).load("dbfs:/FileStore/tables/Legally_Operating_Businesses.csv")
mydf.write.jdbc(url,"license",myproperties)

%scala
import org.apache.spark.sql.SaveMode

mydf.write.mode(SaveMode.Append).jdbc(url,"license",myproperties)

Sur cette page du site, vous pouvez voir la vidéo en ligne databricks connect to SQL db durée heure minute seconde en bonne qualité , qui a été Téléchargé par l'utilisateur Knowledge Sharing 21 mars 2022, Partagez le lien avec vos amis et connaissances, sur youtube cette vidéo a déjà été regardée 25,492 fois et il a aimé 115 téléspectateurs. Bon visionnage!

1,131