When playing with Structured Streams, I left many stream queries active on the back ground. You can see the list with the following command spark.streams.active and you can stop the active queries running spark.streams.active.map(_.stop )
A quick exercise to easily load a csv file with header, into an oracle table, using spark ( spark session created when launching spark-shell ) val prop = new java.util.Properties prop.setProperty("user","username") prop.setProperty("password","password") val vd = spark.read.option("header", "true").option("quote","'").option("escape","^").csv("file:///C:/tmp/test.csv") vd.write.mode("overwrite").jdbc("jdbc:oracle:thin:@host:port/SID","table_name",prop)
Using Regex_replace you can extract bits of a path stored in an Oracle column using grouping backtracking. select regexp_replace('/tsacrm1/inf/prdcrm1/JEE/CRMProduct/application/attachments/attachments/case/3709/Request to clear.txt.txt', '(.+)\/([^\/]+)\/([^\/]+)\/([^\/]+)\.([^\/]+)$','\3') from dual;