Skip to content
DataMiner DoJo

More results...

Generic selectors
Exact matches only
Search in title
Search in content
Post Type Selectors
Search in posts
Search in pages
Search in posts
Search in pages
Log in
Menu
  • Blog
  • Questions
  • Learning
    • E-learning Courses
    • Open Classroom Training
    • Certification
      • DataMiner Fundamentals
      • DataMiner Configurator
      • DataMiner Automation
      • Scripts & Connectors Developer: HTTP Basics
      • Scripts & Connectors Developer: SNMP Basics
      • Visual Overview – Level 1
      • Verify a certificate
    • Tutorials
    • Video Library
    • Books We Like
    • >> Go to DataMiner Docs
  • Expert Center
    • Solutions & Use Cases
      • Solutions
      • Use Case Library
    • Markets & Industries
      • Media production
      • Government & defense
      • Content distribution
      • Service providers
      • Partners
      • OSS/BSS
    • DataMiner Insights
      • Security
      • Integration Studio
      • System Architecture
      • DataMiner Releases & Updates
      • DataMiner Apps
    • Agile
      • Agile Webspace
      • Everything Agile
        • The Agile Manifesto
        • Best Practices
        • Retro Recipes
      • Methodologies
        • The Scrum Framework
        • Kanban
        • Extreme Programming
      • Roles
        • The Product Owner
        • The Agile Coach
        • The Quality & UX Coach (QX)
    • DataMiner DevOps Professional Program
  • Downloads
  • More
    • Feature Suggestions
    • Climb the leaderboard!
    • Swag Shop
    • Contact
      • General Inquiries
      • DataMiner DevOps Support
      • Commercial Requests
    • Global Feedback Survey
  • PARTNERS
    • All Partners
    • Technology Partners
    • Strategic Partner Program
    • Deal Registration
  • >> Go to dataminer.services

How to create an ad hoc data source automation script that connects to Databricks?

503 views18th April 2025Ad Hoc data source Dashboard and Query databricks
4
Zhing Hang Lee672 22nd October 2024 1 Comment

I wanted to create an automation script that connect and fetches data from databricks. Then using these data I will create a dashboard using ad hoc data source.

May I know how I can create this solution? I tried looking at all the documentation and examples but could not find anything that helps.

Thanks.

Marieke Goethals [SLC] [DevOps Catalyst] Posted new comment 18th April 2025
Marieke Goethals [SLC] [DevOps Catalyst] commented 18th April 2025

I see that this question has been inactive for some time. Do you still need help with this? If not, could you select the answer (using the ✓ icon) to indicate that the question is resolved?

1 Answer

  • Active
  • Voted
  • Newest
  • Oldest
2
Ward Haeck [SLC] [DevOps Enabler]3.52K Posted 22nd October 2024 3 Comments

Hi Zhing Hang,

I assume you can access the data you need from Databricks through an API. In that case, you can indeed create an ad hoc data source to fetch that data and use it in your dashboard.

There is a tutorial available which does the same thing, instead of using the Databricks API it uses another API (to fetch data about satellites), but conceptionally it's the same:

Written tutorial: https://docs.dataminer.services/user-guide/Advanced_Modules/Dashboards_and_Low_Code_Apps/GQI/Extensions/Ad_hoc_data/Ad_hoc_Tutorials_Satellites.html

Video format: https://community.dataminer.services/video/kata-4-build-your-first-gqi/

Were you able to check/find these examples (helpful)?

Edit: Do note that the tutorial and video don't go to much in detail about actually fetching the data from the API. But once you have that 'example' downloaded from the catalog, you can take a look at how the implementation works. -> Let me know if you have additional questions.

    private IReadOnlyList<Satellite> FetchSatellites()
    {
        var request = WebRequest.Create(URL);
        using (var response = request.GetResponse())
        using (var stream = response.GetResponseStream())
        using (var reader = new StreamReader(stream, Encoding.Default))
        {
            return ReadSatellites(reader, true);
        }
    }

Hope this helps!

Best regards,

Ward

Ronald Gerard [SLC] [DevOps Advocate] Posted new comment 31st January 2025
Zhing Hang Lee commented 30th October 2024

Hi Ward, thank you for your reply. The links you sent me is somehow useful however as you mentioned, the link did not go into details on the API implementation. I have been struggling mostly with the API implementation part because this example uses a web request and parse in a url, but I need also to parse in a body that contains an SQL statement to the databricks API. Could you advise me how can I do that?

Zhing Hang Lee commented 11th November 2024

Hi, can anyone help to support me on this?

Ronald Gerard [SLC] [DevOps Advocate] commented 31st January 2025

Hi, the actual implementation will be very specific to your use case so it will be difficult for us to explain exactly how you should implement it.

If you are looking for how to parse a response from a databricks API, it will be best to have a look at external resources, like the Databricks API documentation (https://docs.databricks.com/en/index.html) and maybe this Stack Overflow question can point you in the right direction: https://stackoverflow.com/questions/69285974/how-to-read-azure-databricks-output-using-api-or-class-library

Let us know if you have specific questions related to integrating this with our ad hoc data source interface.

Please login to be able to comment or post an answer.

My DevOps rank

DevOps Members get more insights on their profile page.

My user earnings

0 Dojo credits

Spend your credits in our swag shop.

0 Reputation points

Boost your reputation, climb the leaderboard.

Promo banner DataMiner DevOps Professiona Program
DataMiner Integration Studio (DIS)
Empower Katas

Recent questions

Correlation Engine: “Test rule” doesn’t result in a hit, despite functional rule 1 Answer | 3 Votes
When using the Setter = true attribute, will the copy action always be executed first? 1 Answer | 2 Votes
Multiple Set on Table parameters for DVE’s 1 Answer | 2 Votes

Question Tags

adl2099 (115) alarm (62) Alarm Console (82) alarms (100) alarm template (83) Automation (223) automation scipt (111) Automation script (167) backup (71) Cassandra (180) Connector (109) Correlation (69) Correlation rule (52) Cube (150) Dashboard (194) Dashboards (188) database (83) DataMiner Cube (57) DIS (81) DMS (71) DOM (140) driver (65) DVE (56) Elastic (83) Elasticsearch (115) elements (80) Failover (104) GQI (159) HTTP (76) IDP (74) LCA (152) low code app (166) low code apps (93) lowcodeapps (75) MySQL (53) protocol (203) QAction (83) security (88) SNMP (86) SRM (337) table (54) trending (87) upgrade (62) Visio (539) Visual Overview (345)
Privacy Policy • Terms & Conditions • Contact

© 2025 Skyline Communications. All rights reserved.

DOJO Q&A widget

Can't find what you need?

? Explore the Q&A DataMiner Docs

[ Placeholder content for popup link ] WordPress Download Manager - Best Download Management Plugin