close
The Wayback Machine - https://web.archive.org/web/20240927195321/https://www.geeksforgeeks.org/linear-regression-using-tensorflow/
Open In App

Linear Regression Using Tensorflow

Last Updated : 03 Apr, 2023
Summarize
Comments
Improve
Suggest changes
Like Article
Like
Save
Share
Report
News Follow

We will briefly summarize Linear Regression before implementing it using TensorFlow. Since we will not get into the details of either Linear Regression or Tensorflow, please read the following articles for more details:

Brief Summary of Linear Regression

Linear Regression is a very common statistical method that allows us to learn a function or relationship from a given set of continuous data. For example, we are given some data points of x and corresponding y and we need to learn the relationship between them which is called a hypothesis.

In the case of Linear regression, the hypothesis is a straight line, i.e, h(x) = wx + b     Where w is a vector called Weights and b is a scalar called Bias. The Weights and Bias are called the parameters of the model.

All we need to do is estimate the value of w and b from the given set of data such that the resultant hypothesis produces the least cost J which is defined by the following cost function J(w, b) = \frac{1}{2m} \sum_{i=1}^{m} (y_i - h(x_i)) ^ 2     where m is the number of data points in the given dataset. This cost function is also called Mean Squared Error.

For finding the optimized value of the parameters for which J is minimum, we will be using a commonly used optimizer algorithm called Gradient Descent. Following is the pseudo-code for Gradient Descent:

Repeat until Convergence {    w = w – α * δJ/δw    b = b – α * δJ/δb}where α is a hyperparameter called the Learning Rate.

Linear regression is a widely used statistical method for modeling the relationship between a dependent variable and one or more independent variables. TensorFlow is a popular open-source software library for data processing, machine learning, and deep learning applications. Here are some advantages and disadvantages of using Tensorflow for linear regression:

Advantages:

Scalability: Tensorflow is designed to handle large datasets and can easily scale up to handle more data and more complex models.
Flexibility: Tensorflow provides a flexible API that allows users to customize their models and optimize their algorithms.
Performance: Tensorflow can run on multiple GPUs and CPUs, which can significantly speed up the training process and improve performance.
Integration: Tensorflow can be integrated with other open-source libraries like Numpy, Pandas, and Matplotlib, which makes it easier to preprocess and visualize data.


Disadvantages:

Complexity: Tensorflow has a steep learning curve and requires a good understanding of machine learning and deep learning concepts.
Computational resources: Running Tensorflow on large datasets requires high computational resources, which can be expensive.
Debugging: Debugging errors in Tensorflow can be challenging, especially when working with complex models.
   Overkill for simple models: Tensorflow can be overkill for simple linear regression models and may not be necessary for smaller datasets.
Overall, using Tensorflow for linear regression has many advantages, but it also has some disadvantages. When deciding whether to use Tensorflow or not, it is essential to consider the complexity of the model, the size of the dataset, and the available computational resources.

Tensorflow

Tensorflow is an open-source computation library made by Google. It is a popular choice for creating applications that require high-end numerical computations and/or need to utilize Graphics Processing Units for computation purposes. These are the main reasons due to which Tensorflow is one of the most popular choices for Machine Learning applications, especially Deep Learning. It also has APIs like Estimator which provide a high level of abstraction while building Machine Learning Applications. In this article, we will not be using any high-level APIs, rather we will be building the Linear Regression model using low-level Tensorflow in the Lazy Execution Mode during which Tensorflow creates a Directed Acyclic Graph or DAG which keeps track of all the computations, and then executes all the computations done inside a Tensorflow Session.

Implementation

We will start by importing the necessary libraries. We will use Numpy along with Tensorflow for computations and Matplotlib for plotting. 

Python3

import numpy as np
import tensorflow as tf
import matplotlib.pyplot as plt

                    

In order to make the random numbers predictable, we will define fixed seeds for both Numpy and Tensorflow. 

Python3

np.random.seed(101)

                    

Now, let us generate some random data for training the Linear Regression Model. 

Python3

# Generating random linear data
# There will be 50 data points ranging from 0 to 50
x = np.linspace(0, 50, 50)
y = np.linspace(0, 50, 50)
 
# Adding noise to the random linear data
x += np.random.uniform(-4, 4, 50)
y += np.random.uniform(-4, 4, 50)
 
n = len(x) # Number of data points

                    

Let us visualize the training data. 

Python3

# Plot of Training Data
plt.scatter(x, y)
plt.xlabel('x')
plt.ylabel('y')
plt.title("Training Data")
plt.show()

                    

Output:

 

Now we will start creating our model by defining the placeholders X and Y, so that we can feed our training examples X and Y into the optimizer during the training process. 

Python3

X = tf.placeholder("float")
Y = tf.placeholder("float")

                    

Now we will declare two trainable Tensorflow Variables for the Weights and Bias and initializing them randomly using np.random.randn(). 

Python3

W = tf.Variable(np.random.randn(), name = "W")
b = tf.Variable(np.random.randn(), name = "b")

                    

Now we will define the hyperparameters of the model, the Learning Rate and the number of Epochs. 

Python3

learning_rate = 0.01
training_epochs = 1000

                    

Now, we will be building the Hypothesis, the Cost Function, and the Optimizer. We won’t be implementing the Gradient Descent Optimizer manually since it is built inside Tensorflow. After that, we will be initializing the Variables. 

Python3

# Hypothesis
y_pred = tf.add(tf.multiply(X, W), b)
 
# Mean Squared Error Cost Function
cost = tf.reduce_sum(tf.pow(y_pred-Y, 2)) / (2 * n)
 
# Gradient Descent Optimizer
optimizer = tf.train.GradientDescentOptimizer(learning_rate).minimize(cost)
 
# Global Variables Initializer
init = tf.global_variables_initializer()

                    

Now we will begin the training process inside a Tensorflow Session. 

Python3

# Starting the Tensorflow Session
with tf.Session() as sess:
     
    # Initializing the Variables
    sess.run(init)
     
    # Iterating through all the epochs
    for epoch in range(training_epochs):
         
        # Feeding each data point into the optimizer using Feed Dictionary
        for (_x, _y) in zip(x, y):
            sess.run(optimizer, feed_dict = {X : _x, Y : _y})
         
        # Displaying the result after every 50 epochs
        if (epoch + 1) % 50 == 0:
            # Calculating the cost a every epoch
            c = sess.run(cost, feed_dict = {X : x, Y : y})
            print("Epoch", (epoch + 1), ": cost =", c, "W =", sess.run(W), "b =", sess.run(b))
     
    # Storing necessary values to be used outside the Session
    training_cost = sess.run(cost, feed_dict ={X: x, Y: y})
    weight = sess.run(W)
    bias = sess.run(b)

                    

Output:

 

Now let us look at the result. 

Python3

# Calculating the predictions
predictions = weight * x + bias
print("Training cost =", training_cost, "Weight =", weight, "bias =", bias, '\n')

                    

Output:

 

Note that in this case both the Weight and bias are scalars. This is because, we have considered only one dependent variable in our training data. If we have m dependent variables in our training dataset, the Weight will be an m-dimensional vector while bias will be a scalar.

Finally, we will plot our result. 

Python3

# Plotting the Results
plt.plot(x, y, 'ro', label ='Original data')
plt.plot(x, predictions, label ='Fitted line')
plt.title('Linear Regression Result')
plt.legend()
plt.show()

                    

Output:

 


Previous Article
Next Article

Similar Reads

Solving Linear Regression without using Sklearn and TensorFlow
In this article, we will see how can we implement a Linear Regression class on our own without using any of the sklearn or the Tensorflow API pre-implemented functions which are highly optimized for such tasks. But then why we are implementing these functions on our own? The answer to this is very simple that is because they help to clarify our con
3 min read
Multiple Linear Regression using R
Prerequisite: Simple Linear-Regression using RLinear Regression: It is the basic and commonly used type for predictive analysis. It is a statistical approach for modeling the relationship between a dependent variable and a given set of independent variables.These are of two types: Simple linear RegressionMultiple Linear Regression Let's Discuss Mul
3 min read
Linear Regression using PyTorch
Linear Regression is a very commonly used statistical method that allows us to determine and study the relationship between two continuous variables. The various properties of linear regression and its Python implementation have been covered in this article previously. Now, we shall find out how to implement this in PyTorch, a very popular deep lea
4 min read
Pyspark | Linear regression using Apache MLlib
Problem Statement: Build a predictive Model for the shipping company, to find an estimate of how many Crew members a ship requires. The dataset contains 159 instances with 9 features. The Description of dataset is as below: Let’s make the Linear Regression Model, predicting Crew members Attached dataset: cruise_ship_info import pyspark from pyspark
3 min read
Linear Regression in Python using Statsmodels
In this article, we will discuss how to use statsmodels using Linear Regression in Python. Linear regression analysis is a statistical technique for predicting the value of one variable(dependent variable) based on the value of another(independent variable). The dependent variable is the variable that we want to predict or forecast. In simple linea
4 min read
House Price Prediction using Linear Regression | Django
In this article, we'll explore how to use a Python machine-learning algorithm called linear regression to estimate house prices. We'll do this by taking input data from users who want to predict the price of their home. To make things more accessible and interactive, we'll transform this house price prediction code into a web-based system using the
4 min read
Softmax Regression using TensorFlow
This article discusses the basics of Softmax Regression and its implementation in Python using the TensorFlow library. Softmax regression Softmax regression (or multinomial logistic regression) is a generalization of logistic regression to the case where we want to handle multiple classes in the target column. In binary logistic regression, the lab
6 min read
Mathematical explanation for Linear Regression working
Suppose we are given a dataset: Given is a Work vs Experience dataset of a company and the task is to predict the salary of a employee based on his / her work experience. This article aims to explain how in reality Linear regression mathematically works when we use a pre-defined function to perform prediction task. Let us explore how the stuff work
1 min read
ML | Locally weighted Linear Regression
Linear Regression is a supervised learning algorithm used for computing linear relationships between input (X) and output (Y). The steps involved in ordinary linear regression are: Training phase: Compute [Tex]\theta [/Tex]to minimize the cost. [Tex]J(\theta) = $\sum_{i=1}^{m} (\theta^Tx^{(i)} - y^{(i)})^2 [/Tex] Predict output: for given query poi
3 min read
Softmax Regression Using Keras
Prerequisites: Logistic RegressionGetting Started With Keras: Deep learning is one of the major subfields of machine learning framework. It is supported by various libraries such as Theano, TensorFlow, Caffe, Mxnet etc., Keras is one of the most powerful and easy to use python library, which is built on top of popular deep learning libraries like T
9 min read
How To Make Scatter Plot with Regression Line using Seaborn in Python?
In this article, we will learn how to male scatter plots with regression lines using Seaborn in Python. Let's discuss some concepts : Seaborn : Seaborn is a tremendous visualization library for statistical graphics plotting in Python. It provides beautiful default styles and color palettes to make statistical plots more attractive. It is built on t
2 min read
Scatter Plot with Regression Line using Altair in Python
Prerequisite: Altair In this article, we are going to discuss how to plot to scatter plots with a regression line using the Altair library. Scatter Plot and Regression Line The values of two different numeric variables is represented by dots or circle in Scatter Plot. Scatter Plot is also known as aka scatter chart or scatter graph. The position of
4 min read
Orthogonal distance regression using SciPy
Regression basically involves determining the relationship between a dependent variable and one or more independent variables. It generally involves finding the best fit line that minimizes the sum of squared errors for each point. Based on the implementation procedures, regression algorithms are classified as linear regression, ridge regression, l
4 min read
How To Add Regression Line Per Group with Seaborn in Python?
In this article, we will learn how to add a regression line per group with Seaborn in Python. Seaborn has multiple functions to form scatter plots between two quantitative variables. For example, we can use lmplot() function to make the required plot. What is Regression Line? A regression line is just one line that most closely fits the info (in te
1 min read
How to Get Regression Model Summary from Scikit-Learn
In this article, we are going to see how to get a regression model summary from sci-kit learn. It can be done in these ways: Scikit-learn PackagesStats model packageExample 1: Using scikit-learn. You may want to extract a summary of a regression model created in Python with Scikit-learn. Scikit-learn does not have many built-in functions for analyz
3 min read
How to Perform Quadratic Regression in Python?
The quadratic equation is a method of modeling a relationship between sets of independent variables is quadratic regression or we can say the technique of obtaining the equation of a parabola that best fits a collection of data is known as quadratic regression. We use the R square metric to measure the relative predictive power of a Quadratic Regre
2 min read
Plot Multinomial and One-vs-Rest Logistic Regression in Scikit Learn
Logistic Regression is a popular classification algorithm that is used to predict the probability of a binary or multi-class target variable. In scikit-learn, there are two types of logistic regression algorithms: Multinomial logistic regression and One-vs-Rest logistic regression. Multinomial logistic regression is used when the target variable ha
4 min read
Classification vs Regression in Machine Learning
Classification and Regression are two major prediction problems that are usually dealt with in Data Mining and Machine Learning. We are going to deal with both Classification and Regression and we will also see differences between them in this article. Classification AlgorithmsClassification is the process of finding or discovering a model or funct
5 min read
Multivariate Regression
Prerequisite Article-Machine Learning The goal in any data analysis is to extract from raw information the accurate estimation. One of the most important and common question concerning if there is statistical relationship between a response variable (Y) and explanatory variables (Xi). An option to answer this question is to employ regression analys
4 min read
Effect of Transforming the Targets in Regression Model
Regression modelling plays a crucial role in predicting numerical outcomes and understanding the relationships between variables. One key aspect of building robust regression models is the careful consideration of the target variable, as its distribution and characteristics can significantly impact model performance. In this article, we will discus
8 min read
Linear Search Visualizer using PyQt5
In this article we will see how we can make a PyQt5 application which will visualize the linear search algorithm. Linear search or sequential search is a method for finding an element within a list. It sequentially checks each element of the list until a match is found or the whole list has been searched. Gui implementation steps : 1. Create a list
5 min read
Raise a square matrix to the power n in Linear Algebra using NumPy in Python
In this article, we will discuss how to raise a square matrix to the power n in the Linear Algebra in Python. The numpy.linalg.matrix_power() method is used to raise a square matrix to the power n. It will take two parameters, The 1st parameter is an input matrix that is created using a NumPy array and the 2nd parameter is the exponent n, which ref
3 min read
Linear Block Code using MATLAB
Any linear blend of codewords is likewise a code word only. So in coding, a linear code is a mistake correcting code. Linear codes are generally partitioned into block codes and convolutional codes, despite the fact that turbo codes can be viewed as a half breed of these two sorts. Linear codes take into account more productive encoding and deciphe
4 min read
Solve Linear Equations using eval() in Python
Linear equations using one variable of the form a + bx = c + dx can be solved in Python using eval() function. The input type will be a linear equation in the form of a string. Syntax: eval(expression, globals=None, locals=None) Here, we will transform the equation into an expression of real and imaginary numbers such that eval can easily process i
3 min read
Return the Norm of the vector over given axis in Linear Algebra using NumPy in Python
In this article, we will how to return the Norm of the vector over a given axis in Linear Algebra in Python. numpy.linalg.norm() method The numpy.linalg.norm() method is used to return the Norm of the vector over a given axis in Linear algebra in Python. Depending on the value of the ord parameter, this function can return one of the possible matri
3 min read
Return the infinity Norm of the matrix in Linear Algebra using NumPy in Python
In this article, we will how to return the infinity Norm of the matrix in Linear Algebra in Numpy using Python. numpy.linalg.norm() method The numpy.linalg.norm() method returns the matrix's infinite norm in Python linear algebra. This function can return one of eight possible matrix norms or an infinite number of vector norms, depending on the val
3 min read
Solve Two Linear Equations Using Python Sympy
We are given two linear equations and our task is to solve them by using Python Sympy library. In this article, we will cover different approaches to solve two linear equations using Sympy. Solve Two Linear Equations Using SympyBelow are some of the approaches by which we can solve two linear equations using Sympy in Python: Solving Equations with
2 min read
How to Create Custom Model For Android Using TensorFlow?
Tensorflow is an open-source library for machine learning. In android, we have limited computing power as well as resources. So we are using TensorFlow light which is specifically designed to operate on devices with limited power. In this post, we going to see a classification example called the iris dataset. The dataset contains 3 classes of 50 in
5 min read
Deploy a Chatbot using TensorFlow in Python
In this article, you'll learn how to deploy a Chatbot using Tensorflow. A Chatbot is basically a bot (a program) that talks and responds to various questions just like a human would. We'll be using a number of Python modules to do this. This article is divided into two sections: First, we'll train the Chatbot model, and then in section two, we'll l
9 min read
Image Recognition using TensorFlow
In this article, we'll create an image recognition model using TensorFlow and Keras. TensorFlow is a robust deep learning framework, and Keras is a high-level API(Application Programming Interface) that provides a modular, easy-to-use, and organized interface to solve real-life deep learning problems. Image Recognition: In Image recognition, we inp
6 min read
Practice Tags :