Python Programming with Google Colab

In this article, we will learn to practice Python programming using Google colab. We will discuss collaborative programming, automatic setting-up, getting help effectively. Google Colab is a suitable tool for Python beginners.

Introduction

Google Colab is the best project from Google Research. It is an open-source, Jupyter based environment. It helps us write and execute Python based code, other Python-based third-party tools and machine learning frameworks such as Python, PyTorch, Tensorflow, Keras, OpenCV and many others. It runs on the web-browser.

Google Colab is an open-source platform where we can write and Python execute code. It requires any source on the internet. We can also mount it into Google Drive. Google Colab requires no configuration to get started and provides free access to GPUs. It facilitates us to share live code, mathematical equations, data visualization, data cleaning and transformation, machine learning models, numerical simulations, and many others.

Advantages of Using Google Colab

Google Colab provides many features. These features are given below.

• It comes with pre-installed packages.
• We don't need to install Python or its package.
• It runs on the web-browser.
• It allows us to work with web-browser Jupyter notebooks.
• It is an open-source Google platform, which means anyone can use it free of cost.
• It offers GPU and TPU power.
• It supports both Python versions 2 and 3.
• It provides two hardware accelerations - GPU (Graphical Processing Unit) and TPU (Tensor Processing Unit).
• It provides a free Jupyter notebook environment.

What are GPUs and TPUs in Google Colab?

The reason for the popularity of Google Colab it provides free GPUs and TPUs. Machine learning and deep learning training model takes numerous hours on a CPU. We have faced all these issues in our local machine. With GPUs and TPUs, we can train models in a matter of minutes or seconds.

Machine learning enthusiasts always prefer GPU over CPU because of the absolute power and speed of execution. But GPUs are much expensive; not everyone can afford them. That's why Google Colab comes into the scenario. Google Colab is completely free and can continuously run for 12 hours. It is sufficient to meet all requirements of the machine learning and deep learning projects.

Difference between GPU and CPU

A few important features are given below.

Which GPU is used today?

Let's take two scenarios before using the GPU for deep learning.

First Scenario:

First, we need to determine our requirements regarding what kind of resources we need to accomplish our tasks. If our machine learning task is small or can be fit in complex sequential processing, we don't need a big system to work on GPU. It means if we are working on the other ML area/algorithm. We don't require using GPU.

Second Scenario:

If our task is much extensive and has handle-able data then we can use the GPU hardware acceleration. It enhances the execution speed and provides output within seconds or minutes.

Start Working with Google Colab

To start working with Google Colab, we need to type the URL on the web browser below.

It will launch the following window opens with a popup offering many features.

It provides few options to create notebook as well as upload and select from various sources such as - Google Drive, Local computer, GithHub.

Click on the NEW NOTEBOOK button to create new a Colab notebook.

GPU is the best choice. GPUs are much suitable for intensive task.

Uploading File using Goggle Drive

We can upload GitHub's project and search it to get the code.

Similarly, we can upload the code directly from Google drive. We can also filter saved notebooks by name, date, owner, or modified date.

It will automatically upload the code in Jupyter Colab.

Cloning Repositories in Google Colab

We can also create a Git repository inside the Google Colab. We just need to visit any GitHub repository and copy the clone link of the repository.

Step - 1: Find the Github repo and get the "Git" link.

For example - We are using https://github.com/rajk9200/construction.git

Clone or download > Copy the link

Step - 2: Simply, run the following command.

After hitting enter, it will start cloning.

Step - 3: Open Folder in Google Drive.

Step - 4: Open a Notebook and run the Github repo in Google Colab.

Saving Colab Notebook

All the notebooks are saved in the Google Drive automatically after a certain period. Hence we don't lose our working process. However, we can save our notebook *.py and *.ipynb formats explicitly.

Setting up Hardware Accelerator GPU for Runtime

Google Colab offers a free cloud service with a GPU hardware accelerator. For the machine learning and deep learning, it requires very expensive GPU machine. GPU is most essential for doing multiple computations.

Why GPU is important for machine learning

Nowadays, GPU has become the most dominant part of machine learning and deep learning. It provides the optimized capability of more compute-intensive workloads and streaming memory loads.

Another reason of its popularity, it can launch millions of threads in one call. They function unusually better than CPUs although it has a lower clock speed and it also lacks of many core management features compared to a CPU.

Setup Hardware Accelerator GPU in Colab

Let's follow the given step to setup GPU.

Step - 1: Go to the Runtime option in Google Colab.

Step - 2: It will open the following popup screen change None to GPU. We can also select TPU according to our requirements by following the same process.

Step - 3: Now, we will check the details about the GPU in Colab. Type the following code to import the important packages.

Check the GPU accelerator

Output:

/device:GPU:0

Now, we will check the hardware used for GPU.

Output:

[name: "/device:CPU:0"
 device_type: "CPU"
 memory_limit: 268435456
 locality {
 }
 incarnation: 11369748053613106705, name: "/device:XLA_CPU:0"
 device_type: "XLA_CPU"
 memory_limit: 17179869184
 locality {
 }
 incarnation: 514620808292544972
 physical_device_desc: "device: XLA_CPU device", name: "/device:XLA_GPU:0"
 device_type: "XLA_GPU"
 memory_limit: 17179869184
 locality {
 }
 incarnation: 15275652847823456943
 physical_device_desc: "device: XLA_GPU device", name: "/device:GPU:0"
 device_type: "GPU"
 memory_limit: 14640891840
 locality {
   bus_id: 1
   links {
   }
 }
 incarnation: 1942537478599293460
 physical_device_desc: "device: 0, name: Tesla T4, pci bus id: 0000:00:04.0, compute capability: 7.5"]

Using Terminal Commands on Google Colab

We can also use Colab cell for running terminal commands. In the Google Colab, Python libraries such as Pandas, Numpy, scikit-learn.

If we want to install another library of Python using the following command.

It is quite easy to run the command. Everything is similar to the regular terminal. We just need to put an exclamation (!) before writing each command as below.

Or

Sharing Our Notebook

We can share our Colab notebook with others. It is the best way to interact with other data science expert. It helps to share our code same as sharing a Google Doc or Google sheet.

We need to click on the share button. It will show the option of creating a shareable link that we can share through any platform. There is also an option to invite the people through the email address.

It is one of the outstanding features of Google Colab.

Few Important Colab Commands

Colab provides some amazing commands. It offers various commands that facilitate us to perform operations in short. These commands are used with a % prefix. List of all magic commands are given below.

Output:

Available line magics:
%alias  %alias_magic  %autocall  %automagic  %autosave  %bookmark  %cat  %cd  %clear  %colors  %config  %connect_info  %cp  %debug  %dhist  %dirs  

%doctest_mode  %ed  %edit  %env  %gui  %hist  %history  %killbgscripts  %ldir  %less  %lf  %lk  %ll  %load  %load_ext  %loadpy  %logoff  %logon  %logstart  %logstate  %logstop  %ls  %lsmagic  %lx  %macro  %magic  %man  %matplotlib  %mkdir  %more  %mv  %notebook  %page  %pastebin  %pdb  %pdef  %pdoc  
%pfile  %pinfo  %pinfo2  %pip  %popd  %pprint  %precision  %profile  %prun  %psearch  %psource  %pushd  %pwd  %pycat  %pylab  %qtconsole  %quickref  %recall  %rehashx  %reload_ext  %rep  %rerun  %reset  

%reset_selective  %rm  %rmdir  %run  %save  %sc  %set_env  %shell  %store  %sx  %system  %tb  %tensorflow_version  %time  %timeit  %unalias  %unload_ext  %who  %who_ls  %whos  %xdel  %xmode

Available cell magics:
%%!  %%HTML  %%SVG  %%bash  %%bigquery  %%capture  %%debug  %%file  %%html  %%javascript  %%js  %%latex  %%perl  %%prun  %%pypy  %%python  %%python2  %%python3  %%ruby  %%script  %%sh  %%shell  %%svg  %%sx  %%system  %%time  %%timeit  %%writefile

Automagic is ON, % prefix IS NOT needed for line magics.

Command for List of Local Directories

Output:

drwxr-xr-x 1 root 4096 Dec  2 22:04 sample_data/

Command for getting Notebook History

Output:

%lsmagic
%ldir
%history

Getting CPU Time

Output:

CPU times: user 4 µs, sys: 0 ns, total: 4 µs
Wall time: 7.15 µs

Getting the how long has the system has been running

Output:

08:44:32 up 12 min,  0 users,  load average: 0.00, 0.03, 0.03

Display available and used memory

Output:

/bin/bash: -c: line 0: syntax error near unexpected token `('
/bin/bash: -c: line 0: `free -hprint("-"*100)'

Display the CPU specification

Output:

Architecture:        x86_64
CPU op-mode(s):      32-bit, 64-bit
Byte Order:          Little Endian
CPU(s):              2
On-line CPU(s) list: 0,1
Thread(s) per core:  2
Core(s) per socket:  1
Socket(s):           1
NUMA node(s):        1
Vendor ID:           GenuineIntel
CPU family:          6
Model:               63
Model name:          Intel(R) Xeon(R) CPU @ 2.30GHz
Stepping:            0
CPU MHz:             2300.000
BogoMIPS:            4600.00
Hypervisor vendor:   KVM
Virtualization type: full
L1d cache:           32K
L1i cache:           32K
L2 cache:            256K
L3 cache:            46080K
NUMA node0 CPU(s):   0,1
Flags:               fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ss ht syscall nx pdpe1gb rdtscp lm constant_tsc rep_good nopl xtopology nonstop_tsc cpuid tsc_known_freq pni pclmulqdq ssse3 fma cx16 pcid sse4_1 sse4_2 x2apic movbe popcnt aes xsave avx f16c rdrand hypervisor lahf_lm abm invpcid_single ssbd ibrs ibpb stibp fsgsbase tsc_adjust bmi1 avx2 smep bmi2 erms invpcid xsaveopt arat md_clear arch_capabilities

Getting the List of All running VM processes

Output:

List all running VM processes.
Done
error: garbage option

Usage:
 ps [options]

 Try 'ps --help <simple|list|output|threads|misc|all>'
  or 'ps --help <s|l|o|t|m|a>'
 for additional help text.

For more details see ps(1).

How to Design Form in Google Colab

Google Colab provides the facility to design forms. Let's see the following code to understand how to design form in Google Colab. In this section, we will design a form takes the student information.

Output:

Graph Plotting

We can plot various graphs in Colab for the data visualization, as well. In the following example, the graph will show a plot containing more than one polynomial, Y = x³ + x² + x[3]. Let's see the following code.

Example -

Output:

The below code will draw the heat map.

Example -

Output:

TPU (Tensor Processing Unit) in Google Colab

The TPU (Tensor Processing Unit) is used to accelerate on a Tensorflow graphs. They are based on the AI acceleration application-specification integrated circuit which is specially designed for the neural network machine. It is developed by Google.

TPU consists of an excellent configuration of teraflops, floating-point performance, and others. There are 180 teraflops of floating-point performance in each TPU. Generally, a teraflop is the measurement of the computer's speed. Its speed can be a trillion floating-point operations per second.

How to set up a TPU in Google Colab?

The steps are the same as the setup of a GPU.

• Runtime ------> Change runtime

Check Running on TPU Hardware Accelerator

To check the TPU hardware acceleration, it requires the Tensorflow package. Consider the below code.

Running on TPU  ['10.43.45.130:8470']

Conclusion

Google Colab is a Jupyter notebook developed by Google Research. It is used to execute the Python-based code to build a machine learning or deep learning model. It provides the GPU and TPU hardware acceleration and available for free (unless we would like to go pro).

It is quite easy to use and share due to the zero-configuration features requirement.

We can combine the executable code with and HTML, images, Latex, and others. One of the best thing, it has included a vital machine learning library like TensorFlow, already installed. It is a perfect tool for machine learning and deep learning model building.

Colab is outstanding for developing neural networks. The parallelism and execution of multiple threads can be achieved by the CPU based hardware accelerator.

If we use the IDE for machine learning and deep learning model, it will take Nemours hour to execute code. But with the help of the Google Colab, it can be done within few seconds or minutes. It provides the facility to import data from Github and Google drive.