Python Archives - Indium https://www.indiumsoftware.com/blog/tag/python/ Make Technology Work Fri, 26 Apr 2024 11:05:20 +0000 en-US hourly 1 https://wordpress.org/?v=6.5.3 https://www.indiumsoftware.com/wp-content/uploads/2023/10/cropped-logo_fixed-32x32.png Python Archives - Indium https://www.indiumsoftware.com/blog/tag/python/ 32 32 Enhancing Healthcare Solutions with Chat GPT, TDD and Cucumber https://www.indiumsoftware.com/blog/enhancing-healthcare-solutions-with-chat-gpt-tdd-and-cucumber/ Fri, 25 Aug 2023 08:32:52 +0000 https://www.indiumsoftware.com/?p=20476 Introduction Developing software applications involves leveraging technologies and established practices. When building a healthcare solution like an interactive chatbot, integrating Chat-GPT, Test-Driven Development (TDD), and the Cucumber framework enhances development and delivers user-centric solutions. a) Chat-GPT, an OpenAI language model, generates human-like text responses. By integrating Chat-GPT into a chatbot, developers create conversational user experiences

The post Enhancing Healthcare Solutions with Chat GPT, TDD and Cucumber appeared first on Indium.

]]>
Introduction

Developing software applications involves leveraging technologies and established practices. When building a healthcare solution like an interactive chatbot, integrating Chat-GPT, Test-Driven Development (TDD), and the Cucumber framework enhances development and delivers user-centric solutions.

a) Chat-GPT, an OpenAI language model, generates human-like text responses. By integrating Chat-GPT into a chatbot, developers create conversational user experiences using natural language processing.

b) Test-Driven Development (TDD) prioritizes creating automated tests before coding. Following TDD ensures the chatbot functions as intended, meets requirements, and provides expected responses.

c) Cucumber, a popular Behavior-Driven Development (BDD) tool, enables collaboration between technical and non-technical stakeholders. It creates human-readable feature files that describe system Behavior. With Cucumber, developers validate the chatbot, generate tests, and maintain a shared understanding of requirements.

Why do we need integrated technology in healthcare applications?

Without Chat-GPT and Test-Driven Development in the Cucumber framework, the healthcare industry may face several challenges and limitations in terms of communication, efficiency, and quality assurance.

2.1 Let’s understand how these technologies can support and enhance healthcare applications:

Communication Challenges:

  • Limited interactive communication with patients, leading to misunderstandings.
  • Inability to provide personalized responses, resulting in unsatisfactory experiences.
  • Difficulty handling patient inquiries efficiently.

Efficiency Concerns:

  • Manual handling of queries is time-consuming for healthcare professionals.
  • Inability to automate tasks, increasing workload.
  • Potential errors in responses due to human factors.

Quality Assurance Limitations:

  • Inconsistent responses across healthcare professionals.
  • Difficulty validating healthcare applications, leading to bugs and security issues.
  • Inadequate documentation of software requirements.

2.2 Integrating Chat-GPT and Test-Driven Development in the Cucumber Framework addresses these challenges:

Improved Communication:

  • Chat-GPT enables interactive and personalized communication with patients.
  • Test-Driven Development ensures accurate and reliable responses.

Enhanced Efficiency:

  • Chat-GPT automates query handling, freeing up professionals’ time.
  • Test-Driven Development identifies and addresses issues early.

Quality Assurance and Documentation:

  • Test-Driven Development with Cucumber validates application functionality.
  • Cucumber supports clear documentation and a shared understanding of requirements.

This integration enables developers to build intelligent and reliable chatbots, improving user experiences in healthcare and other domains.

How do we integrate the technology into a healthcare application?

Flowchart on the end-to-end process structure

A detailed example of how we can integrate Chat GPT, Test-Driven Development (TDD), and the Cucumber framework in a healthcare solution:

3.1 Define feature files:

Create a feature file using the Gherkin syntax provided by Cucumber.

For example, let’s consider a feature file called “HealthcareChatbot.feature” with the following scenario:

3.2 Write failing step definitions:

3.3 Implement the chatbot system:

To integrate the Chat GPT API using Python, follow these steps:

a.  Set up the project:
• Create a new directory and set up a virtual environment.

b. Install the necessary libraries:
• Install the required Python libraries (e.g., Flask, requests).
• Install the Chat GPT API library, if provided.

c. Create the chatbot module:
• Create a Python file for the chatbot and import the necessary libraries.
• Set up Flask for user interactions.

d. Implement user input processing:
• Define a Flask route to receive user inputs.
• Extract the user’s query from the request and pass the query to the chatbot logic for a response.

e. Integrate with the Chat GPT API:
• Import the API library or module and authenticate with valid API credentials.
• Make a request to the API with the user query and retrieve the generated response.

f. Process and format the API response:
• Extract relevant information from the response in a user-friendly format.

g. Implement the chatbot logic:
• Handle healthcare queries and generate responses.
• Integrate with external healthcare APIs and databases if needed.

h. Connect the chatbot module with Flask:
• Define routes and endpoints for interactions and responses.
• Invoke the chatbot logic and return the response.

i. Test and refine:
• Interact with the chatbot through defined routes for testing.
• Continuously improve based on user feedback and requirements.

j. Deploy and scale:
• Deploy the chatbot on a suitable hosting platform.
• Configure the infrastructure for scalability.

Here is an example of how we can implement the chatbot module that handles user inputs and generates responses based on healthcare-related queries:

 

 

 

 

 

Flowchart on explaining a simplified healthcare chatbot

This is a simplified implementation of a healthcare chatbot in Python. The process_user_input function generates responses based on predefined rules for healthcare queries. If a query doesn’t match the rules, it calls the get_chat_gpt_response function to fetch a response from the Chat GPT API. The get_chat_gpt_response function sends a POST request to the Chat GPT API endpoint with the user query as the prompt.

The API response is extracted and returned as the chatbot’s response. To customize the chatbot, expand the process_user_input function for more healthcare queries and advanced logic. Remember to replace ‘YOUR_API_KEY’ with your actual Chat GPT API key.

Benefits of Integration

1. Engaging experiences: Chat-GPT creates interactive and satisfying interactions.

2. Reliable system: TDD ensures requirements are met and bugs are caught early.

3. Collaborative communication: Cucumber supports BDD for inclusive behaviour definition.

4. Iterative development: TDD enables incremental enhancements and easier maintenance.

5. Clear requirements: Cucumber’s feature files serve as executable documentation.

6. Faster feedback: TDD and Cucumber provide quick issue identification and adjustments.

Combining Chat-GPT with TDD in the Cucumber framework builds a reliable and user-centric chatbot. TDD’s iterative approach and Cucumber’s collaboration enhance development and communication, resulting in realistic and high-quality responses that meet requirements.

Conclusion

The implementation is a healthcare chatbot system that integrates the Chat GPT API for realistic responses. It uses TDD principles with the Cucumber framework. Developed in Python with Flask, the chatbot processes user inputs, integrates the Chat GPT API, formats responses, and implements healthcare logic. The Flask web server handles user interactions, while the chatbot handles healthcare queries and potentially integrates with external healthcare APIs or databases.

The post Enhancing Healthcare Solutions with Chat GPT, TDD and Cucumber appeared first on Indium.

]]>
Power BI Meta Data extraction using Python https://www.indiumsoftware.com/blog/power-bi-meta-data-extraction-using-python/ Wed, 17 May 2023 09:47:06 +0000 https://www.indiumsoftware.com/?p=16850 In this blog we are going to learn about Power BI.pbit files, Power BI desktop file Meta data, Extraction of Power BI Meta data and saving it as an excel file using .pbit file and a simple Python code using libraries like Pandas, OS, Regex, JSON and dax_extract. What is Power BI and .pbix files?

The post Power BI Meta Data extraction using Python appeared first on Indium.

]]>
In this blog we are going to learn about Power BI.pbit files, Power BI desktop file Meta data, Extraction of Power BI Meta data and saving it as an excel file using .pbit file and a simple Python code using libraries like Pandas, OS, Regex, JSON and dax_extract.

What is Power BI and .pbix files?

Power BI is a market leading business intelligence tool by Microsoft for Cleaning, Modifying and Visualizing raw data to come up with actionable insights. Power BI comes with its own data transformation engine called power query and a formula expression language called DAX (Data Analysis Expressions).

DAX gives power BI the ability to calculate new columns, dynamic measures, and tables inside Power Bi desktop.

By default, Power BI report files are saved with .pbix extension which is a renamed version of a ZIP file which contains multiple components, such as the visuals, report canvas, model metadata, and data.

What is Power BI .pbit file

.pbit is a template file created by Power Bi desktop which is also a renamed version of a ZIP file that contains all the Meta data for the Power BI report but doesn’t contain the data itself. Once we extract .pbit file we get a DataModelSchema file along with other files which contain all the Meta data of a Power BI desktop files.

Later in this blog we will be using these .pbit and DataModelSchema files to extract Power BI desktop Meta data.

What is the Meta data in a Power BI Desktop file

Regarding what you see in the Report View in a Power BI desktop, meta data is everything. You can think of all the information as meta data, including the name, source, expression, data type, calculated tables, calculated columns, calculated measures, relationships and lineage between the model’s various tables, hierarchies, parameters, etc.

We will mainly concentrate on extracting Calculated Measures, Calculated Columns, and Relationships in this blog.

Extraction of Meta data using Python

Python was used to process and extract the JSON from the.pbit file and DataModelSchema. We first converted JSON to a Python dictionary before extracting the necessary Meta data.

Below are the steps we will need to achieve the requirement:

 

1. Exporting .pbix file as .pbit file

There are two ways to save our power BI desktop file as .pbit file.

  • Once we are in Power BI desktop, we have an option to save our file as power BI template(.pbit) file
  • We can go to File–>Export–>Power BI Template and save the .pbit file at the desired directory.

2. Unzipping .pbit file to get DataModelSchema file

We can directly unzip the .pbit file using the 7z-Zip file manager or any other file manager. Once we Unzip the file, we will get a folder with the same name as that of the .pbit file. Inside the folder we will get the DataModelSchema file, we will have to change its extension to .txt for reading in python.

3. Reading .pbit and Data model schema file in python

We have an option to directly read the .pbit file in python using the dax_extract library. Second option to read the text file in python and using the JSON module convert it into a Python dictionary. Code can be found in the GitHub repository link given at the end of this file.

4. Extracting Measures from the dictionary

The dictionary that we get consists details of all the tables as separate lists, Individuals tables have details related to the columns and measures belonging to that table, we can loop on each table one by one and get details of columns, Measures etc. Below is an example of the Python code can be found in the GitHub Repository link given at the end of this file.

  table Number table Name Measure Name Measure Expression
0 5 Query Data % Query Resolved CALCULATE(COUNT(‘Query Data'[Client ID]),’Quer…
1 5 Query Data Special Query Percentage CALCULATE(COUNT(‘Query Data'[Client ID]),’Quer…
2 6 Asset Data Client Retention Rate CALCULATE(COUNT(‘Asset Data'[Client ID]),’Asse…

 

5. Extracting calculated columns from the Dictionary

Like how we extracted the measures we can loop on each table and get details of all the calculated columns. Below is the sample output of the Python code can be found in the GitHub Repository link given at the end of this file.

 

  table no Table Name name expression
6 2 Calendar Day DAY(‘Calendar'[Date])
7 2 Calendar Month MONTH(‘Calendar'[Date])
8 2 Calendar Quarter CONCATENATE(“Q”,QUARTER(‘Calendar'[Date]) )
9 2 Calendar Year YEAR(‘Calendar'[Date])

 

Also Read:  Certainty in streaming real-time ETL

6. Extracting relationships from the dictionary

Data for relationships is available in the model key of the data dictionary and can be easily extracted. Below is the sample output of the Python code can be found in the GitHub Repository link given at the end of this file. 

 

  From Table From Column To Table To Column State
0 Operational Data Refresh Date LocalDateTable_50948e70-816c-4122-bb48-2a2e442… Date ready
1 Operational Data Client ID Client Data Client ID ready
2 Query Data Query Date Calendar Date ready
3 Asset Data Client ID Client Data Client ID ready
4 Asset Data Contract Maturity Date LocalDateTable_d625a62f-98f2-4794-80e3-4d14736… Date ready
5 Asset Data Enrol Date Calendar Date ready

 

7. Saving Extracted data as an Excel file

All the extracted data can be saved in empty lists and these lists can be used to derive a Pandas data frame. This Pandas data frame can be exported as Excel and easily used for reference and validation purposes in a complex model. Below snapshot gives an idea of how this can be done.

Do you want to know more about Power BI meta data using Python? Then reach out to our experts today.

Click here

Conclusion

In this blog we learnt about extracting metadata from .pbit and DataModelSchema file. We have created a Python script that allows users to enter the file location of .pbit and DataModelSchema file and then metadata extraction along with excel generation can be automated. The code can be found on the below GitHub also sample excel files can be downloaded from below GitHub link. Hope this is helpful and will see you soon with another interesting topic.

 

The post Power BI Meta Data extraction using Python appeared first on Indium.

]]>
Why Python is the Language of the Future https://www.indiumsoftware.com/python-programming-language-future Mon, 28 Jun 2021 06:18:50 +0000 https://www.indiumsoftware.com/blog/?p=3973 Why is Python the Language of the Future? From the time of its release in 1991 to today, Python has evolved, and as of February 2021, documentation for version 3.9.2 had been released. It is the third most popular language used by developers and its tutorials are the most searched on the Net. A SlashData

The post Why Python is the Language of the Future appeared first on Indium.

]]>
Why is Python the Language of the Future?

From the time of its release in 1991 to today, Python has evolved, and as of February 2021, documentation for version 3.9.2 had been released. It is the third most popular language used by developers and its tutorials are the most searched on the Net. A SlashData report published in ZDNet indicates there were 8.2 million Python developers in 2019, up from 7 million the previous year. Compare this to Java, which had 7.1 million users in 2018 and 7.6 million in 2019.

There are many reasons for Python’s popularity, and the increasing search for tutorials indicates that this is going to be the future too.

5 Reasons for Python’s Popularity

Today, speed of development has become essential to shorten time to market and retain competitive advantage. Another emerging trend is machine learning, which helps automate tasks and speed up processes across industries, freeing up resources to add value. Python fits the bill for both.

Reduce your application development time drastically

Read More

Python is a flexible, versatile, multi-purpose, object-oriented and high-level programming language that can be used for developing a variety of applications across platforms such as Windows, Mac, Linux, and even Kivy platform to develop games for the desktop and web applications. The Django or Flask frameworks also enable Python developers to create web-based applications that are scalable and can interact with popular databases such as MySQL and PostgreSQL.

Some of the reasons for its popularity include:

  1. The Simple Language: Python uses English keywords that make it easy to learn and use. It takes a modular approach with a high degree of code readability and is an interpreted language that does away with the need for a compiler to run it.
  2. Versatility: Its modular nature enables developers to create packages for different applications such as NumPy for working with numbers, matrices and vectors; SciPy for technical and engineering computations; Pandas for data manipulation and analysis; and Scikit-Learn for AI-related operations.
  3. Pseudocode: Python allows the use of pseudocode, an algorithm that does not conform to specific syntax rules, enabling the developer to focus on the query rather than the language. This simplistic and minimalistic feature facilitates dealing with common programming tasks easily.
  4. Open Source: Python is free and open source with a large community that has been instrumental in developing the language as well as third-party libraries. Developers can access pre-written code and standard libraries, further reducing the need for coding for some common features and increasing Python’s usefulness as a development tool.
  5. Python Tools: Python developers can also benefit from a large array of tools such as Tkinter–a GUI development tool, custom python interpreters and support, internet protocol, file formats, built-in functions, and modules, among others.

3 Reasons Why it is the Language of the Future

The ease and convenience make it a popular choice for developers today. But its future seems just as bright due to the following reasons:

Artificial Intelligence

Python rules the disruptive and futuristic world of digital transformation, especially Big Data, artificial intelligence and machine learning today. Be it speech recognition or data analysis, Python is the driving force behind AI’s success today, powering the Internet of Thingy devices as Python or Micropython. Its built-in libraries such as NumPy, Pandas, and SciPy empower data scientists with lots of functionalities they need for tasks such as data manipulation and data visualization. Using Python can speed up advanced calculations to move from hypothesis to data analysis quickly. Matplotlib is a rich library from Python, making it ideal for data manipulation and visualization.

Web Development

Python’s versatility makes it very ideal for web development as it can help you get started quickly and remain productive with every new function and feature. Python offers two tools, Flask for a minimalist, flexible framework, and Django for a fairly rigid structure that facilitates prototyping and getting up and running quickly.

Networking

Python programming language is popular for developing and implementing programs to configure routers and switches, facilitating networking and automation duties in a secure and cost-efficient manner.

The fact that Python is here to stay and travel with us in the future is evident from its use cases across industries. Right from NASA’s Workflow Automation System (WAS) developed using Python to Google’s APIs, report generation and other data related features, YouTube

video streaming, Disney’s scripting language for most of its animation-related tasks and production, Python is powering the next-gen applications. Facebook, Instagram, Netflix, Quora, and Spotify are among some of the other users of Python.

If you are interested in reducing your time to market for your digital transformation efforts, we at Indium Software can help you analyze the best-fit use of Python and get you up and running quickly.

Future-Proof Your Development with Python

Today, speed, effectiveness and efficiency are the three key pillars to success and growth. Python facilitates this by speeding up the development process and allowing for easy scaling and flexibility.

Leverge your Biggest Asset Data

Inquire Now

Indium Software’s team of developers with experience and expertise in a variety of languages including Python can provide end-to-end solutions for your digitization needs. Our solutions go beyond languages and cover:

Thus, we integrate knowledge in Python with digital transformation to make you future-ready. To know more about Indium and how we can help you, contact us now:

The post Why Python is the Language of the Future appeared first on Indium.

]]>
Key differences between R and Python (Infographic) https://www.indiumsoftware.com/blog/r-vs-python/ Wed, 09 Sep 2020 05:11:19 +0000 https://www.indiumsoftware.com/blog/?p=3344 R Python Usage The purpose of R is to develop a language that focused on delivering a more user-friendly way to perform data analysis, statistics and graphical models. It is an object-oriented programming language, which means it groups data and codes into objects that can interact with and modify one another. It allows developers to

The post Key differences between R and Python (Infographic) appeared first on Indium.

]]>
tr:nth-child(odd) td { background: #f3f3f3; } td, th { border: 1px solid #dddddd; text-align: left; padding: 8px; } tr:nth-child(even) { background-color: red !important; }
R Python
Usage The purpose of R is to develop a language that focused on delivering a more user-friendly way to perform data analysis, statistics and graphical models. It is an object-oriented programming language, which means it groups data and codes into objects that can interact with and modify one another. It allows developers to execute tasks with better stability, modularity, and code readability.
Statistical analysis R was built to do statistical and numerical analysis of large data sets, so that you’ll have many options while exploring data with R. statsmodels in Python and other packages provide decent coverage for statistical methods.
Dashboard Shiny library which allows for creating rich interactive web apps. It is recommended for complex and customized applications. Shiny library which allows for creating rich interactive web apps. It is recommended for complex and customized applications.
Big data R is considered slow. It requires its objects to be stored in a physical memory. R is considered slow. It requires its objects to be stored in a physical memory.
Data Analysis R is convenient for analysis due to the huge number of inbuilt packages, readily usable tests and the advantage of using formulas. R is the more efficient language for task. The Python packages for data analysis were an issue but this has improved with the recent versions. Numpy and Pandas are used for data analysis in Python.
Machine learning & Deep Learning.

For ML problems Caret being a very popular one and R has many packages for ML.

The Nnet library is a suitable platform for modeling neural networks.

If a machine learning program requires a wide range of operations, R may present some constraints.

Python has popular ML packages in today’s world. Scikit-learn presents users with several fundamental tools for creating machine learning models.

For creating DL models packages such as tensorflow , pytorch, theano are more efficient and robust when compared with R.

Time series R is better choice for Advanced Hidden Markov models, Hierarchical time series forecasting and advanced econometric models. Python is a better choice for LSTM and other deep learning models for time series.
NLP R doesn’t offer great built-in string manipulation. A specific Rword2vecs package exists to provide word2vec model, while the LDA is available within the MASS package. Python has a diverse ecosystem supporting NLP. NLTK that provides a library for preprocessing. For modelling it offers Gensim that provides both LDA and word2Vec models.
Object-relational mapping (ORM): R has none Python has two well used ORM libraries in Django ORM and SQL Alchemy.
Authentication & Authorization R lacks proper support for these kinds of features. Setting up an authentication & authorization layer on an API is done by Flask login, or Auth0 in Python.
Data Acquisition R has much less coverage, rest call is being supported by a mixture of httr & jsonlite, while soap calls require a mixture of the Rcurl and XML package. For GraphQL there are few wrapper libraries . Python provides great support for API calls, Rest API calls are handled by the requests library, while SOAP call are usually handled using Zeep, and offers a few clients for GraphQL.
Web Scraping R leverages Rvest, a library part of the Tidyverse. Python handles web scraping through the BeautifulSoup library
In-database computation R is Supported by SQL Server, Postgres, Teradata and Oracle. Python has fewer options, it is supported by Postgres, Microsoft’s SQL Server.
Workflow Automation R doesn’t have a Workflow automation tool written in its language. Python has two key workflow automation tools, Airflow and Luigi.
SDK R on the other hand has no such support or have to use unofficial package library such as cloudyr that contains SDKs features. On AWS it is supported through the Boto SDK, on azure python is supported by a SDK . Google Cloud also supports through a SDK client.
Cloud-ML Services R is supported by multiple cloud service for instance Azure ML Services and Google CloudML service. Python is supported by all cloud platforms, having integration on AWS, Azure ML Services, Sagemaker and Google Cloud.
Serverless No official support currently exists for R on these platforms Python has serverless function support for AWS Lambda, Azure functions or Google Functions.

Share this Image On Your Site

The post Key differences between R and Python (Infographic) appeared first on Indium.

]]>