Hi everybody, have you guys prepared for Christmas and New Year?

Bad news. My roommate finally gets a girlfriend and he will hang out with her on Christmas. Thus, I will stay at home alone that day, watch some movies of Woody Allen (believe me, this man is so funny, but maybe not funny enough to help me overcome the day) and then sleep with pain and hopelessness. What a tragedy.

OK, just a little talk, I don’t want to continue imagining that day. Let’s come back to the topic I will talk about today: Using Matplotlib library to visualize data or to plot some graphs.

First of all, I need to say that Matplotlib is the oldest visualization library for Python. It was designed to closely resemble MATLAB so you could plot most kinds of graphs supported in MATLAB with Matplotlib though it is not as beautiful as graphs plotted by MATLAB. Nonetheless, you don’t have to worry much about it, not long ago (maybe nearly 1 year ago), Matplotlib was upgraded (I don’t remember exactly which version) and now, it could be able to make better visualization.

Actually, there are other libraries that provide the visualization much better than Matplotlib. However, it still is a good way to get basic visualization from learning Matplotlib.

As usually, before rolling up your sleeves to explore the visual world of Matplotlib, you need to install it first.


Just like installing Numpy in the previous tutorial (PyLIB 2), there also are 2 ways to install Matplotlib.

The first way is to access to this link and download the version of the library that is suitable for your Python version and operating system.

The second way is to use the command “pip”

pip install matplotlib

for Python version 2.*, or

pip3 install matplotlib

for Python version 3.*.

Note that if you decide to use “pip”, Matplotlib and Matplotlib-toolkits will be installed automatically. On the other hand, if you choose to install Matplotlib manually, please remember to download and install Matplotlib-toolkits on your computer too or you will get some troubles with 3D plotting.

2D Graphs

There are many kinds of scientific graph supported by Matplotlib, by which, you could get a clear view of your data, algorithm progress, or result from analysis. Here are several types I would like to introduce to you.

Firstly, I would like to talk about the line graph and some basic plot methods.

Secondly, I want to introduce to use bar graph and how to use the module subplots.

Thirdly, here is a little bit about scatter plot used for visualization of classification tasks on Machine Learning.

Finally, at the of this section, I want to show you how to display an image using Matplotlib.

3D Graphs

The original Matplotlib doesn’t support 3D plotting, however, you could draw some 3D graphs by using Matplotlib-toolkits.

That’s all I want to talk about Matplolib today but not all properties of this library. You could do more miracle things as you want (such as create animation or stimulate some phenomena) by exploring the Matplotlib documentation. However, because of the enormous number of the methods supported by Matplotlib and the limit of my purpose that is aimed to introduce only methods that are often used in Machine Learning, I could not cover more.

In the next post, I will introduce to you another scientific library which is often used along with Numpy named Scipy.

Now, at the end of the post, let Sam Smith shine.


Enjoy your holiday.

Merry Christmas and Happy New Year,

Curious Chick



Hi everybody,

To continue our series of Python libraries, today, I would like to introduce to you Numpy which is also known as one of the famous scientific computing libraries in Python. Some of you may be familiar with MATLAB before and know how efficient it is when performing calculations with matrices.

Numpy helps us do the same by supporting tools to manage and manipulate multi-dimensional arrays such as matrices. It plays a vital role in the field of computing. Most of the time, when I was working with Machine Learning, I often used Numpy (along with Scipy sometimes) to manipulate the data (such as concatenate two set of data or remove outliers) before fitting it into a model. Of course, there are other libraries help you process the data easier. One of the examples of this is Pandas which is one of the emerging stars in financial data analysis. However, I was familiar with MATLAB when I was a student in an engineering school and feel more conventional when using Numpy. Moreover, I assure you that if you are familiar with Numpy, it will not be difficult for you to turn to use another kind of scientific computing libraries such as Pandas mentioned above.

In fact, Numpy is used for not only linear algebra but also signal processing and statistics. However, these two last are beyond the scope of this tutorial, and I will talk about them later in another.

Now, before rolling up your sleeves to explore how miracle Numpy is, we first need to install it.


The version of Python I use in this tutorial is 2.7.11. However, you can also use all syntaxes for others, regardless the version of 3.* or 2.*.

To install Numpy, there are 2 ways:

Using source file

The first is to access the link: https://sourceforge.net/projects/numpy/files/NumPy/. Then download and install the version that is suitable for your Python version (2.* or 3*) and your operating system (Window, Linux, or MacOS).

Using “pip”

The second way is to use “pip” command. If you have not yet been familiar with “pip” or don’t know what it is. It is OK. Please refer my tutorial on how to install and use “pip” in Python.

Now, open your command window and run this command:

pip install numpy

for Python version 2.* or this command

pip3 install numpy

for Python version 3.*.

If it is not successful for Linux. Try to use

sudo pip install numpy


sudo pip3 install numpy

for the Python version 2.* or 3.*, respectively.

Basic linear algebra with Numpy

First, try to create some vectors and matrices yourself with Numpy.

Next, use some Numpy built-in methods to create some special matrices.

Then, let’s try to use some operators (such as addition, subtraction, or dot product) between two matrices.

And other performance like concatenate two arrays or reshape one.

That’s all about basic linear algebra with the Numpy library that I would like to talk about, by which, you could do a lot of things in data analysis. Of course, what presented above is not all of Numpy. There is more and more miracle that the Numpy can do such as analyzing statistics or processing a signal. For more details, you could find and read its documentation.

In the next tutorial, you will learn how to visualize data and plot some basic graphs (such as the line or bar chart) by using a popular library named Matplotlib. In this post, I promise we will draw some fun with math functions I found on the Internet. Until then, hope you enjoy my posts.

See ya,

Curious Chick


Hi everybody, how are you doing?

What I am gonna present today is another project of mine on Brain/Human-Computer Interface (abbreviated as BCI). In this topic, I will not only introduce my project but also talk a little bit more about the fundamental (like what it is, how it works, and why I chose it as a method for my project).


If you watch TV, follow some Tech pages on the Internet, or read Tech news every day, you will see that BCI systems, time by time, have proved its ability to bring significant benefits to daily-life applications, especially in supporting handicapped people as an alternative communication system. There are two types of BCI that are frequently used for controlling in real life are motor imagery based and Steady-state visual evoked potential (abbreviated as SSVEP).

Motor imagery based BCI

The operation of the former one is based on the characteristics of our sensorimotor area.

To illustrate how it could be used in BCI system, let’s take a look at the video above. At first, we need to know what we are watching. The two images on the top show the mapping of C3 (left) and C4 (right) electrodes on the scalp. Now, look at the two graphs below, it describes the change of the power of the alpha rhythm (at C3 and C4) when an event-related potential (ERP) occurs (left-hand or right-hand imagery movement in this case). There is one more point that you need to notice is the relation between the top mappings and the bottom graphs. If the power of alpha at the electrodes increase, the color of the area around the electrode position will turn to more red, or else, it will turn bluer. In the next step, let’s me explain the phenomenon that is the fundamental of motor imagery based BCI system. Now, let’s take a look at the left part, when the left-hand imagery movement occurs, as we can see in the left-bottom graph, the power of alpha at the position C4 decrease significantly. In fact, concurrently, the power of beta at the position C4 increase considerably, however, it is not illustrated in the video. By looking at the mapping on the right-top of the video, we can see the right area turn to the background color whilst the left are still remained its one. Then, we could ask ourselves, why the left-hand imagery movement affect the power of alpha and beta on the right? It can be explained by the mechanism of the crossover connection in the brain (as shown in Fig. 1).


Fig. 1 The crossover connections of left hand to right hemisphere, right hand to left hemisphere [1]

There are many BCI systems were built based on this approach and many research were conducted over the world for the purpose of improving the performance and increasing the accuracy (or the classification result) of the motor imagery based BCI system. However, non-invasive BCI systems based on motor imagery is still not reliable and robust to apply to multi-task (i.e. three or more) in daily-life apps. Recently, some research groups worldwide have developed some invasive one though, it still not possible to apply to the real life because of the cost and risks of brain surgery.


Regarding the latter, EEG signal used in this type of BCI generated from visual cortex (Fig. 2) when it reacts to visual stimulation (such as a flickering LED at specific frequencies).


Fig. 2 Visual cortex area [2]

The principle of this type of BCI is that when you’re looking at a flickering sign or LED at a specific frequency, your brain will generate an electrical activity that has the same (or multiple of) frequency of visual stimulation [3]. For example, if we look at an 8 Hz visual stimulation (as I’ve said before, it could be a flickering LED), your brain will generate steady-state potentials at 8, 16, and 24 Hz (Fig. 3).


Fig. 3 Response of brain to visual stimulation of 8 Hz [4]

Here is another video that shows an application of SSVEP based BCI in controlling a game.

We can see that, in a BCI system based on SSVEP, there are more than one visual stimulations could be used to adapt to the design condition. Each visual one is assigned to a state or control command that can be classified by using some popular classifiers (random forest, support vector machine or neural network).

The signal generated by using the SSVEP approach has a high signal-to-noise ratio and relative immunity to artifacts (like EMG, ECG, or EOG). As a result, it could be used in real-life applications reliably and robustly. However, though it could cope the disadvantage of the former one (i.e. reliability and robustness in multi-task), it can cause some annoyance to users because of exposure to flickering LED for a long time of use.


In my project, I proposed a simple approach for BCI application by combining it with IoTs. The project is implemented for the purpose of supporting the handicapped people, or more exactly, supporting families with disabled people in caring their relatives who have disabilities. The Fig. 4 shows my overall BCI system.


Fig. 4 Overall BCI-based IoTs application

In the project, I used EPOC that is an inexpensive EEG device to acquire the signal from the user’s scalp. Then, EEG signal is translated to control commands by the BCI module. These commands will be received by IoTs module after that to help the handicapped people signal to their relative (as Fig. 5(a)) or send an SOS message (in case of nobody is at home as Fig. 5(b)) to show that they are in urgent need of help.


Fig. 5 The use of our IoTs application

To help the user to use our application easily, we design a GUI that is shown in Fig. 6.


Fig. 6 Graphical user interface

And then, we build a prototype for the experiment.


Fig. 7 The prototype of the IoTs module


In my project, I decided to build the BCI system based on motor imagery because of two main reasons. Firstly, as I said before, even though the SSVEP approach is more reliable to use in daily life, it could cause some eye problems if it is used day by day. Secondly, using the BCI system based on motor imagery seems to be more natural than that based on SSVEP. The reason of that is, normally, before getting stroke or injuries, people often use their arms to reach their purpose, for example, they want to take a cup to make tea. Now, after accidents, people could also think of their limbs too, but to signal to get help from the others instead.



Fig. 8 Development in future

For further developments in future, it would be more effective if we can embed the ability of emotion recognition. On the other hand, the families with the disabled would also need to coordinate with medical organizations to support for taking care the disabled better.

If you are interested in my project, please feel free to contact me. I will be glad to share ideas and receive feedback from you.


Thank you and see you,

Curious Chick



[1] https://www.joshuanava.biz/memories/the-double-brain.html

[2] https://www.slideshare.net/lasitham/brain-anatomy-12098255

[3] http://synaptitude.me/blog/a-quick-intro-to-ssvep-steady-state-visually-evoked-potential/

[4] https://www.intechopen.com/books/advances-in-robot-navigation/brain-actuated-control-of-robot-navigation



Hi everyone,

So far, we have known how to use basic control structures to perform sequential, looping and selecting tasks. In this tutorial, to take a further step, I would show you the way to help you to manage your program easier when the number of lines in your code become larger and to make the program more concise.

The first thing I wanna show you is functions. It can be seen as blocks of code written to execute a specific task. A function can return whether values (like numbers, strings, lists, etc.) by using the command return or nothing (for example, functions that are used just to display information).

How to create a function

To create a function, we use the structure below:

def name_of_the_function(parameters):                      (1)
   statements                                              (2)

   (return value)                                          (3)

Remember, just like when using for-looping structures or if-structures, we need an indentation to make the Python interpreter understand which parts of code belong to the function.

Elements of a function

OK, let’s take a look at the structure of the function above.

In the line (1), the command def help the interpreter know that you are defining a function, followed by the name of the function. In addition, a function could have parameters or not. In the next section, I will show you more detail about how to pass arguments into a function.

In the line (2), we put statements here to perform desired tasks.

The line (3) shows that a function could return a value or not, depending your purpose when writing it.

Together, let’s make a ‘Hello World’ program, but now, we will write a function to do it instead of only using the command print.

The say_hello above is a function that doesn’t return any values, its purpose is just to print the line “Hello World” on the screen.

Here is another example, in which, a function return a value of an addition.

Or you can even return a list.

All the codes above can be downloaded from my github: https://github.com/tenga061/basic_python_tutorials/blob/master/pytut7_functions/lab1_elements_of_a_function.py.

Parameters of a function

Now, it’s time to take a further detail in parameters in function and how to pass arguments into it.

Now, let’s tweak the ‘Hello world’ program above by adding a parameter ‘name’ to it. Our program now won’t say hello world when we call it but say hello to the name that we have entered, instead.

By that way, we added a parameter and passed argument to our function.

The code can be downloaded from my github: https://github.com/tenga061/basic_python_tutorials/blob/master/pytut7_functions/lab2_say_hello.py.

Passing a fixed number of arguments

Let’s take a look at another example below. The code show a function using Euclid algorithm to find the greatest common divisor of two nonnegative m and n.

Functions that we made above is kind of functions with fixed number of parameters. Because of this, be careful when calling a function with parameters, the number of arguments you pass need to be equal to the number of parameters, otherwise, errors will occur.

You can download the code from my github: https://github.com/tenga061/basic_python_tutorials/blob/master/pytut7_functions/lab3_euclid_gcd.py.

Passing an arbitrary number of arguments

So, to have a function that permit you to pass the arbitrary number of arguments, there is two ways.

The first one is to put all values into a list and pass the list to the function.

The second is to put the symbol ‘*’ before the parameters of the function.

The codes can be downloaded from the link: .

Using default values

Sometimes, you want to set the default value for the parameters.

For example, back to the ‘Hello World’ example above. Now, we set the default value of parameter ‘name’ of the function to ‘World’, so, if we don’t pass any value into the function when calling it, it will print ‘Hello World!’ again, otherwise, it will say hello to the name we have passed.

The code can be downloaded here: https://github.com/tenga061/basic_python_tutorials/blob/master/pytut7_functions/lab5_say_hello_2.py.

The scope of variables

The last thing in this post I wanna tell you is that how vital to know the scope of variables.

Let’s continue to learn from codes together.

You can download the code here: https://github.com/tenga061/basic_python_tutorials/blob/master/pytut7_functions/lab6_scope_of_variable_1.py.

So, you can see the results clearly. The variables a and b which are initialized in the function check_print just exist while the function was running.

Now, if you wanna use the variables a and b as global variables, how could you do that? What if we don’t declare variable names in the function like the code below?

It’s OK, but don’t do that frequently, it sometimes brings to you serious mistakes in your code. To use global variables, I prefer to put ‘global’ before the name of variable in the function.

The code can be downloaded here: https://github.com/tenga061/basic_python_tutorials/blob/master/pytut7_functions/lab7_scope_of_variable_2.py.

OK, that’s all I wanna talk about in this post, in the next tutorial, I will introduce basic of classes in Python.

Hope you enjoy it,

Curious Chick


Hi everyone!

First of all, I would like to say thank you to all my friends and people who support me to make this series of Python tutorial. Two weeks ago, I shared the series with some of my friends and got many positive comments from them and they encouraged me to continue further. I would say that your concern really is my motivation to continue to do this.

OK now, let’s take a look at last tutorials a bit, in which, I introduced how to do some basic math with numbers, play with strings and three types of Python array, then assign values to variables.

To take a further step, in this tutorial, I want to show you control structures in Python.

By the way, I set up a new account on Github and from now, all codes will be published to the link below.


And all codes in this tutorials were published on:


As usual, I recommend you to read the following topics before beginning this tutorial.

Related topics


There are three types of control structures as you can see in Fig. 1. The first one is sequence structure that you used in the preceding tutorials. The second and last one is selection and loop structure, respectively.


Fig. 1. Flow charts of three types of control structures [1]

In most programs, the two last structures proved to be helpful to make the codes more concise or more clear. But now, before beginning with them, let’s try using sequence structure again to make a simple program that print numbers from 0 to 9 on the screen but using Python script editor instead of Python Shell (or Python interpreter prompt).


To open Python script editing window, do the following steps:

  • First, you need to open IDLE (read [PyTUT 1] AN INTRODUCTION OF PYTHON again if you don’t remember how to open IDLE).
  • Second, press CTRL + N  or left click on File -> New File on IDLE tool bar in order to open the Python script window.


Fig. 2 Open IDLE script
  • Third, in Python script editing window, press CTRL + S or left click on File -> Save and name the file to save the script. Actually, it is unnecessary to save the script when you begin to code, but you ultimately must do this if you want to run the script.


Fig. 3 Save script file

Following to this, we will start to write a first script that print the numbers from 0 to 9.

print 'The number from 0 to 9'
print 0
print 1
print 2
print 3
print 4
print 5
print 6
print 7
print 8
print 9

After that, press F5, then select OK when you are asked whether you wanna save this program or not in order to run the script (see Fig. 4).


Fig. 4 Save all changes in your code before running

The result is showed in Fig. 5.


Fig. 5 The result was showed on Python Shell

This code is available on: https://github.com/tenga061/basic_python_tutorials/blob/master/pytut6_control_structure/lab1_sequence_structure.py.

As you can see, in sequence structure, the computer will start to run the program from the begin to the end and follow the statements in order. However, imagine what if you want to print the numbers from 0 to 10,000. OK, it is still possible if you have a lot of time, right? Believe me, you will not want to do this.

For the reason of this, in the next section, I will introduce to you repetition structures which will help you to solve such repetitive problems.


As I mentioned before, repetition structures are useful for repetitive problems. There are three types of repetition structures, including: for-loop, while-loop, and do-while-loop. First, look at the for-looping structure as showed in Fig. 6.

For-Looping Structure

The flow chart in Fig. 6 was taken from a C++ book but I think it’s still good to help you understand the mechanism of the for-looping structure in Python or even in any other programming languagues you want to learn in the future.


Fig. 6 For-Looping Structure [1]

In C++, to create a for-loop, you first need a initial statement to initialize a loop variable. Then, declare a loop condition such as i < 10, where i is a loop variable. If i satisfies the condition (i < 10), statements in the loop will be performed and the loop variable will be updated after that (like i = i + 1 or i = i + 3, etc.). By contrast, the computer will close the loop and perform another statement.

The mechanism of for-loop in Python is similar to C++. However, if you were not familiar with C++ before, it still is OK. Let’s take a look at the example below that print numbers from 0 to 10 using the for-looping structure in Python:

for i in range(0, 10):
   print i

As you can see, to create a for-loop in Python, we first initialize a loop variable i, just like C++. However, there is a little bit different from C++. In Python, to create a loop condition, we use range(m, n) function which make a list from m to (n-1) with n>m, the upper bound n of the range function is implied as the loop condition i < (n-1). Additionally, the loop variable will automatically update itself in every value in the list in every loop (see Fig. 7 to understand how it works).


Fig. 7 The way the for-loop structure work in Python

Now, what if you want to print only odd numbers from 0 to 9? Fortunately, you can add step argument into the range(m, n, step) function. So, the code become:

for i in range(1, 10, 2):
   print i

(This script is available on: https://github.com/tenga061/basic_python_tutorials/blob/master/pytut6_control_structure/lab2_for_loop.py)

The results are illustrated in Fig. 8.


Fig. 8 For-looping structure example results

Not only those, for-looping structure of Python is even more flexible, by which, you can loop to print arbitrary numbers as you want by assigning the value of each element in the list to loop variable.

arbitrary_numbers = [9, 123, 59, 1039, 23, 27]
for number in arbitrary_numbers:
   print number

You can even do this for tupe:

lovely_cities = ('Paris', 'Los Angeles', 'Seoul', 'Tokyo')
print 'Here is some of the lovely cities: '
for city in lovely_cities:
   print '   + ' + city

Or dictionary type:

hp_characters = {'Danial Radcliffe': 'Harry Potter', 
                 'Emma Watson': 'Hermione Granger',
                 'Rupert Grint': 'Ron Weasley'}
for actor, character in hp_characters.iteritems():
   print actor + ' played ' + character + ' in the series of Harry Potter'

OK, you saw how useful is it when using for-loop to solve repetitive tasks. But you can only use the for-loop if you know exactly how many time it loop. For some problems that you don’t now the number of loop, you need to use while-looping structures which will be introduced in the next section.

While-Looping Structure


Fig. 9 While Looping Structure [1]

As I mentioned before, in while-loop, you don’t needa know how many times your loop will run but loop conditions such as i \leq n or i \neq n . In the case, we use while-looping structure.

While-looping structure begin by checking the condition loop or expression like i \leq n . Then, if the condition is satisfied, statements in the loop will be executed. Otherwise, the loop will be ended. Note that, of course we can’t type symbol like \geq or \neq in any programming languages. Instead, we use these signs in the table below.

Table 1. Relational operators [1]


For example, we want to write a program that read arbitrary numbers from user. If the number is equal or less than 9, the program is continue, by contrary, it stop and print the line “Goodbye and see ya”.

user = 1

while (user<=9):
   a = raw_input('Please enter an arbitrary number: ')
   user = float(a)

print 'Goodbye and see ya'

The code can be downloaded here: https://github.com/tenga061/basic_python_tutorials/blob/master/pytut6_control_structure/lab3_while_loop_game_1.py.

Now, what if you wanna use more than one looping condition, for example, the program will continue requiring you to enter an arbitrary number if your entered-number is equal or less than 9 and more than 6. In the case, we needa use boolean operator and.

user = 1

while (user>6) and (user<=9):
 a = raw_input('Please enter an arbitrary number: ')
 user = float(a)

print 'Goodbye and see ya'

Similarly, you can use boolean operator or in Python by typing or. It’s little bit different for people who were familiar with C++ or other lower programming language, in which, they must use && for the boolean and and || for the or. So, please note that, in Python, we just needa type and for the and and or for the or.

Do-While-Looping Structure


Fig. 10 Do-While-Looping Structure [1]

The only difference between while-loop and do-while loop is that statments in the loop will be excuted at least one. In detail, if the looping condition is wrong from the begining of the loop, the while-loop will end without any statements executed, while statements in do-while-loop still is executed once because it is executed before looping conditions are checked.

Nested Looping Control Structures

Just like other programming languages, we can use looping structures in looping structures to handle more complicated tasks. You can write a simple code that paint a Christmas tree using only for loop. Here is my sample:

for m in range(3):
   for i in range(1, 5+m):
      for j in range(5-i+1):
         print ' ',
      for k in range(5-i+1, 5+i):
         print '*',

And the result:


Fig. 11 The result of the Christmas tree program

You can download the code from my github link: https://github.com/tenga061/basic_python_tutorials/blob/master/pytut6_control_structure/lab4_christmas_tree.py.

Python loop controls

In this part, I would like to introduce some special commands that help you handle your loop somtimes.


As its name, it mean ‘don’t do anything’. It sounds pointless, but sometimes we use it with the loop to wait for an occurrence of an event or an interupt.

Here is an example of using pass command in a loop:


The program above will run until you close the termial.


This command is another way to end looping structures whenever it satisfy the condition that you set.

For example,

i = 0
while 1:
   print i
   i = i + 1
   if i > 5:

Running the program and you will get the result as illustrated in Fig. 12.

Theoretically, the program should run and print the value of i till we close the terminal. However, because we set a break in the if block (will be introduced in the next section), the break break the loop when i = 6.


Fig. 12 An example of using break for while-loop

You can also use the break command in for-loop. The example below using for-loop but show the same result.

for i in range(10):
   if i > 5:
   print i

When I was an undergraduate student, my professor who taught me the subject ‘Programming Fundamentals’ often prohibit us from using this command because, as he said, the break break the structure of programs. However, I think this control command sometimes is useful, especially, it is used frequently in programming for microcontroller. Anyway, you should consider carefully if you want to use it in your program.


This command returns the control to the beginning of the loop. Let’s run the following program to see how it work.

i = 0
while i<10:  
   i = i + 1
   if i == 5:
   print i

The result is described in Fig. 13.


Fig. 13 An example of using the command continue in a program

We can see that the command continue in the program above is the reason why we miss the number 5. In details, when i reach the value 5, the if condition is satisfied, then the command continue is run. As a result, the control of the loop go back to the begining without execute the line print i (i.e print i = 5 on the screen).


Now, we turn to the last structure of this tutorial – selection or if-structure. In the previous section (Python loop control), I briefly show you using the selection structure to make condition for breaking the loop. Now, we introduce further.

One-Way Selection

The simplest form of selection structures are one-way selection. In which, we use an if condition or expression to decide whether statements should be executed or not. In selection structures, the expression is described as the same way as the expression of while-loop in Table 1.


Fig. 14 One-Way selection [1]

Let’s me show you an example in the normal life. Assume that you go to the market and want to buy a fish for lunch. You go to a stall and ask the price of a fish, if it is lower than $10, you buy it, but if not, you don’t. Here is a code:

print 'How much money for this fish?'
fish_price = raw_input("It's: $")
fish_price = float(fish_price)
if fish_price>10:
   print 'Sorry, I don\'t wanna buy it'

Now, run the program, enter 20 for the cost of the fish, you will get the result ‘Sorry, I don’t wanna buy it’.

Multiple Selection


Fig. 15 Two-Way selection [1]

From the view of real life problems, you can see that it’s not simple like that. Here is another script: You go to the supermarket, ask the price of the fish, if the price less than $10 then you buy it, if not, you deal with the salesman that if you could buy it with the price of $13.

To do the script above we need to know how to use the multiple selection structure. Here is the structure of two-way selection (as described as Fig. 15)

if condition:

If you need more than 2, we use elif for conditions and else for the last one.

if condition_1:
elif condition_2:
elif condition_n:

Here is the script of the problem above.

print 'How much money for this fish?'
fish_price = raw_input("It's: $")
fish_price = float(fish_price)
if fish_price<10:
   print 'OK, give me one.'
   print 'Could I buy it with $13?'

Nested Selecting Control Structures

Just like repetition structures, you can put selection structures into others. Here is a small example of using nested selecting control structures.

Just for fun

OK, so, I introduced you the foundation of control structures in Python. Now we can use it to make a simple entering-password code. In which, users are asked to enter their password, if the entered password is right, the program will show them where were  the mythical treasures hide, if not, the program will end with curse: ‘You have wrong answer for 5 times. Now, you will not be able to eat candy. Ha ha ha!’.

You can download the code from my github: https://github.com/tenga061/basic_python_tutorials/blob/master/pytut6_control_structure/lab5_mythical_treasure.py.

So this is the end of this topic. In the next, I would introduce you function, class and package in Python. Goodbye and see you again.

Hope you enjoy it,

Curious Chick


[1] D. S. Malik, C++ Programming: From problem analysis to program design (Sixth Edition), Cengage Learning (2013)


Hi everyone! It has been a long time no see. I got extremely busy preparing my application for the deadline of Spring 2018 semester admission at universities to study at a higher level of education and do my further researches on the field of Machine Learning and Human-Computer Interface. It really was a difficult process, occupied most my time for travelling around and around the city for preparing necessary paperworks, writing statments like statment of purpose or study plan and sending emails to my professors to ask for recommendation letters.

Related Topics:


OK, just a little talk, to begin this topic, let’s remember, I introduced you how to implement a Linear Neuron Model to solve a linear problem with the aid of famous library Tensorflow. However, remember that how frequently do you face with the linear problem? Honestly, it rarely happens, even the “Materialism” problem related to the number of girl friend and monthly salary we dealt with in the first and second topic of “Linear Regression” is also not a pure linear problem. Let me show you what is a problem you get when using linear model for this. You might say that it was very good at trying to guess the number of my may-be-have girl friends (GF) in the first topic. But how about trying to use this linear model to predict the number of girl friends of the richest man in the world – Bill Gates (OK, this is just a joke, I mean no offence when I said that). From [1], we know Bill’s monthly income is \$216,666,666.00 , infer from the model, we get the number of GF he may have is 0.45753154 + 0.00087198*216,666,666.00 = 188,929.457 . How could it be real? Theoretically, may be. Practically, it’s infeasible, even Casanova couldn’t reach this number. Now, you see the problem here, if the monthly income of a man increase to infinite, the number of GF he has will increase to infinte too while the limit on the number of population worldwide or others such as morality, religion, etc. make it impossible. To deal with this, we need a restriction on the upper bound.


First, we needa add a little bit more in our dataset to make a restriction. Assume that, even though there is a man who earn more than $15,000 monthly, the maximum number of GF he has are not higher than 10.


Fig. 1 The new dataset of the relation between monthly income and the number of GF of a man

The dataset can be downloaded from my github link: https://github.com/tenga061/machine_learning/blob/master/linear_regression/episode_3/dataset/funny_dataset.csv.

For this dataset, we won’t try to guess the number of Bill’s GF because, as you know, we don’t have enough data (our dataset stop at $20,000). Instead, we will implement a polynomial model next, and then use it to test another called testing dataset.

Polynomial regression model

Polynomial function

For our dataset, the polynomial model has form:

\hat{y} = f(x, \omega) = \omega_{0} + x_{1} \omega_{1} + x_{1}^{2} \omega_{2} + ... + x_{1}^{n} \omega_{n} \indent (1)

Or we can rewrite it into:

\hat{y} = f(x, \omega) = \sum_{i=0}^{n}{x_{1}^{i} \omega_{i}}

Or by matrix form:

\hat{y} = f(x, \omega) = \bold{X} \omega \indent \indent \indent \indent (2)

Let’s look at the equation (1), f(x, \omega) is a nonlinear function of x though, it is a linear function of the coefficient \omega . Because of that, we can still implement the polynomial model as the same way as we do it with the linear one in the first topic. But now, we have the form of the matrix \bold{X} is:

\bold{X} = \begin{bmatrix} 1&x_{1}&x_{1}^{2}&...&x_{1}^{n}\\ 1&x_{2}&x_{2}^{2}&...&x_{2}^{n}\\..\\1&x_{k}&x_{k}^{2}&...&x_{k}^{n} \end{bmatrix}

Implementing polynomial model using TensorFlow

To form input vector after feeding it into the model, I wrote the function below.

Forming X data:

# Creating the input matrix X
def create_x(order):
   global mans_monthly_income

   # Creating element vector
   X = np.ones((len(mans_monthly_income)), 1)
   for i in range(1, order+1):
      temp = np.power(mans_monthly_income, i)
      x_temp = np.array(temp, dtype=float)
      x_temp = np.transpose(np.matrix(x_temp))

      X = np.column_stack((X, x_temp))
   return X

Forming y data:

# Creating the target vector y
def create_y():
   global total_number_of_gfs
   y = np.array(total_number_of_gfs, dtype=float)
   y = np.transpose(np.matrix(y))

   return y

The results are showed below when using polynomial function with order of 3, 7 and 9, respectively:


Fig. 2 The model with order of 3


Fig. 3 The mode with order of 7


Fig. 4 The model with order of 9

The code can be downloaded from my github: https://github.com/tenga061/machine_learning/blob/master/linear_regression/episode_3/code/12_lr_nonlinear_polynomial.py.

Testing the model

The question now is that how could we know which order is the best? To answer this, I made another dataset (red points in Fig. 5) for the purpose of testing our model by evaluating the root-mean-square (RMS) error (showed as Eq. (3)) each order from 0 to 10.

E_{RMS} = \sqrt{\frac{1} {N} \sum_{i=1}^{N} {(\hat{y} - y)^{2}}} \indent \indent \indent \indent (3)


Fig. 5 Training (blue points) and testing (red points) dataset

The testing dataset can be downloaded here: https://github.com/tenga061/machine_learning/blob/master/linear_regression/episode_3/dataset/funny_dataset_test.csv.


Fig. 6 The relation between the value of RMS error and the order of polynomial function

The code can be downloaded from my github: https://github.com/tenga061/machine_learning/blob/master/linear_regression/episode_3/code/13_lr_nonlinear_polynomial_rms.py.

From the Fig. 6, we can see that the RMS value of models with order higher than 8 begin increasing. In that case, we say our model was overfitted. By which, the model couldn’t work well with other test datasets (even it may work wrong), though it have done well with the training. To deal with this problem, there is some techniques can be used.


Increasing the size of the dataset

The first is to collect more and more data. This depends on the data acquisition step, sometimes we can, but sometimes not. There is a ‘lex non scripta’ supported by many is that the number of data points should be larger than some multiple (from 5 to 10 is OK) of the number of parameters in the model.


However, what happens if you are in the situation that you cannot acquire more data. In the case, I would like to introduce you a simple technique of regularization by adding \frac{1} {2} \lambda \omega^{2} to the loss function \mathcal{L} (\omega) .  Now, the loss function become:

\mathcal{L}(\omega) = \frac{1} {2} \sum_{i=1}^{N} {(y_{i} - \bold{x}_{i})^{2} \omega} + \frac{1} {2} \lambda \omega^{2} \indent \indent \indent \indent (4)

The lambda in the adding element help us penalize the loss when the model become overfitting and keep the parameters small.

Turn Eq. (4) to the matrix form:

\mathcal{L}(\omega) = \frac{1} {2} (\bold{y} - \bold{X} \omega)^{2} + \frac{1} {2} \lambda \omega^{2} \indent \indent \indent \indent (5)

Calculating the derivation of Eq. (5) and set it equal 0 to find roots of the model:

\frac{\partial{\mathcal{L} (\omega)}} {\partial{\omega}} = -\bold{X}^{T} (\bold{y} - \bold{X} \omega) + \lambda \omega = 0


(\bold{X}^{T} \bold{X} + \lambda \bold{I}) \omega = \bold{X}^{T} \bold{y}

Assume that (\bold{X}^{T} \bold{X} + \lambda \bold{I}) is a nonsingular matrix, we have parameters of model is the root of the equation:

\omega = (\bold{X}^{T} \bold{X} + \lambda \bold{I})^{-1} \bold{X}^{T} \bold{y}

Implementing polynomial with regularization using Scikit-Learn lib

The part of code to implement the model with regularization using Scikit-Learn is showed below:

# Creating the model and feeding data to it
model = linear_model.Ridge(alpha=lamda)
model.fit(X, y)

# Getting the coefficients
coef = model.coef_
params = coef[0]
print params

The code can also be downloaded from my github: https://github.com/tenga061/machine_learning/blob/master/linear_regression/episode_3/code/16_lr_regularization_sklearn.py.

Implementting polynomial with regularization using TensorFlow lib

The part of code to implement the model with regularization using TensorFlow is showed below:

# Creating constant opt to store data
X_tensor = tf.constant(X)
y_tensor = tf.constant(y)

''' Finding the root of the model '''
# Calculating Xt*X
first_step_1 = tf.matmul(tf.transpose(X_tensor), X_tensor)
# Calculating lamda.I
temp_step = (order+1)*lamda*np.identity(order+1)
first_step_2 = tf.constant(temp_step)
# Calculating Xt*X + lamda.I
first_step = tf.add(first_step_1, first_step_2)
# Calculating (Xt*X + lamda.I)^(-1)
second_step = tf.matrix_inverse(first_step)
# Calculating (Xt*X + lamda.I)^(-1)*Xt
third_step = tf.matmul(second_step, tf.transpose(X_tensor))
# Calculating (Xt*X + 2.lamda.I)^(-1)*Xt*y
root = tf.matmul(third_step, y_tensor)

''' Getting parameters '''
with tf.Session() as sess:
   params = sess.run(root)

print params

The code can also be downloaded from my github: https://github.com/tenga061/machine_learning/blob/master/linear_regression/episode_3/code/15_lr_regularization.py.

Sigmoid function

Another way to confront with nonlinear problems is to use functions having features suiting dataset features. This is based on your experience much.

Look at our dataset, on the one hand, we can see points distributing in a curve that is partly look like a function called “sigmoid” (as showed as Fig. 7).


Fig. 7 The graph of sigmoid function

On the other hand, we have a restriction on the upper bound, so it can be easily recognised that these features suit sigmoid’s features well. For the reason, now, we will use the model showed in Eq. (6) instead of the polynomial model as before.

\hat{y} = f(x, \omega) = \omega_{0} + \frac{1} {1 + e^{-x}} \omega_{1} \indent \indent \indent \indent (6)

However, note that the input x is very large in our dataset (up to 20,000), so e^{-x} is very small and can be set as 0. For that reason, we need to add a trick to the Eq. (6) by dividing x by a constant number k. In this case, I use k = 4000, the Eq. (6) turn into:

\hat{y} = f(x, \omega) = \omega_{0} + \frac{1} {1 + e^{-x/4000}} \omega_{1} \indent \indent \indent \indent (7)

The new model is illustrated in Fig. 8.


Fig. 8 The sigmoid model

The code can be downloaded from my github: https://github.com/tenga061/machine_learning/blob/master/linear_regression/episode_3/code/9_lr_nonlinear.py.

Apart from polynomial or sigmoid function, we can also apply others like trigonometic or logaric function for our regression model.

So, that’s all I want to talk about how to use linear model for regression tasks. In the next topic, I would introduce the way to use linear model, but for classification tasks.

Hope you enjoy it, goodbye, and see ya,

Curious Chick


[1] https://paywizard.org/main/salary/vip-check/world-richest-people

[2] Simon Rogers, Mark Girolami – A first course in Machine Learning

[3] Christopher M. Bishop – Pattern Recognition and Machine Learning

[4] http://scikit-learn.org/stable/modules/linear_model.html


Many people who watch sci-fi film a lot may be familiar with the idea of using brain to control devices or even people. In fact, this have been partly done thanks to developments in neuroscience and computational methods. Of course, till now, scientists have not yet pioneered any technology which can be able to be used to control other people. But at least, Brain-Computer interface (BCI) – a recently developed technology relating to the classification of brain waves – helps us to control some electrical and electronic devices, which bring sharp benefits to people, especially the disabled one. For example, the disabled can use BCI to activate their wheel chair or switch some household devices like TV, air conditioner, etc. without any help from others. On the other hand, BCI technology can also be used for educational purposes or analyzing decision making of customers in fields of economy such as sale and marketing, too name but a few. From these, it can be said that, BCI has been becoming one of sexiest disciplines in the world, along with machine learning and internet-of-things trends.

Limitations notwithstanding, BCI still is an awesome technology in the belief that, by which, we can make superpowers real or become superheroes in the near future. With this belief, I tried to convince my professor to allow me to build a non-invasive BCI system for my thesis, and after a while of deliberation, he agreed. The result is showed in the video below.


From an old story,

Curious Chick