Friday, September 22, 2017

"AI and The Future" by Margaret Boden

Last Tuesday, I attended to an interesting guest talk by Professor Margaret Boden from University of Sussex under the topic AI and The Future. Even though I'm not much into Artificial Intelligence topic, I decided to attend to this session simply driven by curiosity. Most importantly, this is the first session of its kind I'm attending in UCD and therefore I just wanted to see how guest talks are going to be here. The talk was delivered at the Moore Auditorium in UCD Science Center.
In her talk, she mostly talked about the implications of AI systems in our everyday life and what she believes as the things to be worried about. The concern she pointed out in her talk is that, when AI systems replace human presence from certain activities, we tend to receive bad results and bad long term impacts on our society even though AI application can seem to be useful and makes life easier. In order to support her point, she took a broad range of examples including autonomous drones, AI systems to take care of elderly and kids.

Even though I have a habit of posing a question when I attend to sessions like these, unfortunately, I couldn't ask a question in this session. They were running out of time and many people wanted to ask questions. However, I feel that it's worth the time I invested on attending this guest talk. I will hopefully attend to the future sessions organized by them.

Wednesday, August 16, 2017

Demodulating AM Signals using GNURadio

I have been using GNURadio for different purposes for some time. However, in most of those times, I used either a very simple flow graph created by myself or a complicated flow graph created by somebody else. Therefore, in the case of complicated flow graphs, I didn't even bother to explain how they work as I didn't really understand their internal functionality. Yesterday, I came across a need to demodulate a signal captured from an RTL-SDR and saved to a file. From the output I observed through the GQRX tool, the signal is modulated using Amplitude Modulation (AM). I wanted to implement a demodulator by myself and therefore I searched the web for a tutorial regarding this kind of stuff. I found a tutorial given in the references section which was really useful [1].

In the rest of this post, I will be explaining how this AM demodulation flow graph is implemented and the functionality of each block according to my understanding. There can be missing details and inaccurate things. If you notice some glitch, please be kind to let me know by leaving a comment. Here we go.

(1) The signal I'm trying to demodulate is in 32MHz frequency and therefore, I tuned an RTL-SDR dongle to that frequency and captured data to a file with a sample rate of 1.8MHz. The tool I used for this purpose is a command line tool called osmocom_fft which comes in handy when we need to save captured data into the .cfile format.

(2) Let's start with a File Source block in the flow graph. Double click on it and then browse to select the source file which contains data. Set the Repeat option to yes.

(3) The data in the file is recorded in 1.8MHz sample rate and the samples are centered to the 32MHz signal. If we draw an FFT spectrogram from the raw data of the file, we will see the X-axis from (32-1.8)MHz to (32+1.8)MHz. However, the real AM signal does not span such a width. The real signal was within about 5kHz width (not exactly). Therefore, let's narrow down our data scope to somewhere about 10kHz in order to focus on our signal. To narrow down our spectrum information from 1.8MHz width to 10kHz, we need to resample the data. So, let's use a Rational Resampler block as the next step.

In the Rational Resampler block, we have two important parameters; Interpolation and Decimation. Interpolation value represents the number by which the input signal's sample rate is multiplied. Decimation value represents the number of which the input signal's sample rate is divided. In order to resample a 1.8MHz signal into the 1kHz signal, we have to divide the input by 180 and multiply by 1. That means, set the Interpolation to 1 and Decimation to 180 in the Rational Resampler block.

Here, instead of directly putting the resampling value 18 in the field, let's make the value a variable so that later we can change it easily. Therefore, add a variable called resamp_factor and set the value to 180.

(4) It is important to note that the purpose of the Rational Resampler is to change the sampling rate of some data. The output of the current setup is a signal with a sample rate of 10kHz. However, to actually isolate the signal we need to extract, we should filter out the other frequencies and keep the target frequency. This is where we need a filter. The target signal is about 5kHz wide. And it is centered at the 35MHz. Therefore we need to add a Low Pass Filter block with a cutoff frequency at 5kHz. Any signals which contain more than 5kHz components will be removed by this block.

When setting the sample rate of the Low Pass Filter block, it is important to set it to the value (samp_rate / resamp_factor) since, the sample rate is now changed by the previous block, the Rational Resampler.

(5) Now we are focused to the exact signal we need to demodulate. Since we are interested in amplitude modulation (AM), we are interested in the amplitude of the signal. We can convert the complex data we were receiving out of the filter to a series of numbers which represent the amplitudes using a Complex to Mag module. Now the output of this module is basically the demodulated base band signal.

(6) It is useful to have a way to increase and decrease the amplitude of the demodulated signal in order to listen to it conveniently. In order to achieve that, we add a Multiply Const block. In the constant field, instead of adding an exact number, we put a name called volume. Then, we should add WX GUI Slider block separately. Set the ID of this new block to be volume. Set the minimum value 0 and maximum value 10 with a number of steps set to 100. Now, when we change the volume in the WX GUI slider, it will change the value in the Multiply Cont block.

(7) Before we add an audio sink to listen to the sounds, we need one more thing. Since an audio sink would need the sample rate to be around 48kHz, let's resample the sample rate of the current setup from 10kHz to 48kHz by using another Rational Resampler. Set the decimation to 10 and interpolation to 48 for the obvious reasons.

(8) Now the last thing. Add an Audio Sink block and set the sample rate to 48kHz. If you think it's useful, add a WX GUI Scope Sink which will show the waveform of the audio signal being heard. The sample rate of that block too has to be 48kHz.

Run this flow graph and listen to the output sound. You should be able to hear the demodulated signal. In case you need to take a look at the original GNURadio Companion based file I created for the above AM demodulator, download the GRC file in the GIST given in this link.  Cheers!



Sunday, August 6, 2017

Memories of Staff Development Program at University of Kelaniya

While being a lecturer in University of Colombo School of Computing, I had a requirement of following a staff development program as one of the conditions to get confirmed in my position. Even though I have expertized in my own field of study, there's no doubt that I'm not a complete person as a lecturer. How to properly involve in the teaching and learning process as a lecturer is something I have to learn separately. The university grants commission (UGC) of Sri Lanka has accredited several staff development programs (SDC) offered in different universities in Sri Lanka. Among these, I selected the SDC program in University of Kelaniya for me.

Since, I did my bachelors degree in University of Colombo and joined the staff of the same university, I had never got a chance to participate to a course in another university in Sri Lanka as a student. I have been to University of Sri Jayawardanepura, but as a visiting lecturer, not as a student. Therefore, me joining the SDC program in University of Kelaniya is the first time I played the student role in another university. Prof. Dilkushi Wettewe who was the director of the SDC program then, assisted me so much when I contacted her to get further details. Even though University of Kelaniya is located a little bit far away from University of Colombo, I considered it has a positive change to visit a new place and be a student. It indeed became a whole new experience.

The course was scheduled to be in every Friday and the course goes from the morning to the evening across the whole day. The participants to the course included new lecturers from various universities in Sri Lanka such as University of Kelaniya, University of Vocational Technology and Bhikkhu University Anuradhapura. I was the only participant from University of Colombo. Before attending to the course, I never realized that I have so much to learn about teaching and learning process as a lecturer. The course starts with the introduction to the university system in Sri Lanka and how the parliamentary acts have formally established the universities in Sri Lanka. Throughout the course, we studied various aspects of university education. The most exciting thing I came across by attending to the SDC program is the participants I got to know there. I believe that the contacts I made with the people will remain for the rest of my carrier.

At the last day of the course, we had a little ceremony to award the certificates to the attendees of the course where I performed the vote of thanks. I was lucky to receive a best portfolio award which was awarded only to 5 participants of the course. I'm really happy about the opportunity I got to take part in the program.


Thursday, July 27, 2017

Inspectrum: A tool to inspect signals from SDR

If we are using a software defined radio (SDR) device to capture some signals and planning to analyze it later, we might need a good software to visualize the data. The most obvious solution is to use GNURadio Companion itself as it has the required modules to plot waveform, FFT and waterfall graphs. However, today I found another useful tool which we can use to visualize and inspect recorded signals. It is called Inspectrum. Following are the steps to easily install it and try using a dummy data file generated using GnuRadio Companion.

sudo apt-get update
sudo apt-get install inspectrum
sudo apt-get install gnuradio

Now, let's start GnuRadio Companion by using the following command on the terminal.


Create the flow graph shown in the following figure. It reads from a sample Wav file we have stored in our computer and writes it to a new file in the correct format which we need. The file extension of the file we write to should be .cfile in order for the Inspectrum tool to recognize it properly. Let's say our output file name is mydata.cfile.

Flow graph to generate sample data

It's time to try the new tool we wanted to see. Run the following command in the terminal to start Inspectrum with the data file we created.

inspectrum mydata.cfile

Inspectrum window with sample data file

Monday, July 17, 2017

Writing in Sinhala (සිංහල) using Latex (a follow up post)

Sometime back, I wrote a blog post about preparing Latex documents with Sinhala language [1]. I received very good feedbacks for that post and encouraged me to explore more. However recently, I came across a question posted in Stack Exchange regarding Latex documents in Sinhala [2]. The nice thing was that my original blog post was mentioned in that thread. I jumped in and tried to help the person who raised the question. This attempt helped myself to understand another important thing which is about how to use any Sinhala font we want in our Latex documents.

In this article, I decided to write down those simple steps of getting a Sinhala Latex document with our favorite Sinhala font for my future reference. Here we go.

(1) First of all, we need Latex with all the packages installed in our computer. I'm using a 64-bit Ubuntu 16.04 LTS machine. I can use the following command on the terminal to install Latex with all the packages.

sudo apt-get install textlive-full

(2) Now, open your favorite text editor and create a file with the name my-example.tex and save it in some directory (folder). This is the file where we will put our text contents.

(3) Now, you need to put the Sinhala font file in the directory. I'm going to use the fond called Iskoola Potha. It comes as a file with the file extension ttf. If you want the same file I used, you can download it from here [5]. I renamed the font file name to be iskoola-pota.ttf.

(4) Now, everything is set for us to write our Sinhala contents in the Latex file. Open the my-example.tex file and put the following content there.


\usepackage[a4paper,left=2.5cm, right=2.5cm, top=2.5cm, bottom=2.5cm]{geometry}

\setmainfont[Extension=.ttf, Language=Sinhala, BoldFont=iskoola-pota, ItalicFont=iskoola-pota]{iskoola-pota}

\title{මාතෘකාව මෙතන ලියන්න}
\author{අසංක සායක්කාර}




මේ මම සිංහලෙන් සකස් කල ලිපියක්. මෙය සකස් කිරිඉමට මම \bf{ලේටෙක්} මෘදුකාංගය පාවිච්චි කලා. එය ඉතාම ප්‍රයෝජනවත් මෙවලමක් මට.


Some English here and there.


(5) It's time to generate the PDF file and see how it looks like. Run the following command from the terminal for that.

xelatex my-example.tex

Now, you may notice that there's a PDF file generated in the local directory where you had your Latex and TTF files. Open the PDF file to see your output.

Saturday, April 29, 2017

OpenSSL on Linux Terminal for Cryptography

OpenSSL is the most important library we will be using if we are to use some cryptographic functions on a Linux computer. There are hundreds of variations of cryptographic algorithms and related functions are available on Linux terminal with OpenSSL. In this post, I would like to note down few useful things we can do with OpenSSL library on Linux.

Base64 Encoding:

When dealing with various files and data types, sometimes it is useful to convert everything into text format. Base64 is an encoding technique which we can use to convert different data into text format and then decode from text format to the original format. Let's see how we can do it.

First of all, let's select an image file in jpg format. We are going to encode this image file in base64 format as follows.

cat old-house.jpg | openssl enc -base64 >> encoded-image.jpg

Now, it you try to open the encoded image file, you will see that you cannot open in as a usual image file. However, the nice thing is, you can open the encoded image file using a text editor because the content is printable characters now. Let's decode the encoded image file back to the original format.

cat encoded-image.jpg | openssl enc -base64 -d >> decoded-image.jpg

Data Encryption Standard (DES):

DES is a little bit older data encryption algorithm which we don't have to use nowadays. Anyway, with OpenSSL library, we can use DES as follows to encrypt data.

openssl enc -des -a -in plantext.txt -out ciphertext.des

Once encrypted, we can use the following command to decrypt the data back to the plain text.

openssl enc -des -d -a -in ciphertext.des -out plaintext.tex

Advanced Encryption Standard (AES):

AES is a better and stronger encryption algorithm compared to DES. We can encrypt a file using AES and then decrypt the file back to the plain text using the following two commands.

openssl aes-256-cbc -in plaintext.txt -out ciphertext.aes

openssl aes-256-cbc -d -in ciphertext.aes -out plaintext.txt

Rivest, Shamir, Adleman (RSA):

When we are browsing web on our computer or mobile phone through a secure channel, the encryption algorithm which helps us most importantly is RSA. It is an asymmetric key encryption algorithm where we have two keys called as Public key and Private key. Private key is what we always keep to ourselves while the Public key can be distributed among everybody.

First of all, we need to generate a key pair in order to use RSA. Let's generate the private key and public key using the following two commands.

openssl genrsa -out private_key.pem 1024

openssl rsa -in private_key.pem -out public_key.pem -outform PEM -pubout

If you check the local directory where you were when running the above commands, now you should be able to see that two new files are created as private_key.pem and public_key.pem which are our keys. Now, we can encrypt data using RSA. Let's say we are going to encrypt a text file so that only a friend can see it. Then, we have to use the friend's public key during the encryption as follows.

openssl rsautl -encrypt -inkey public_key.pem -pubin -in plaintext.txt -out ciphertext.txt

Now, when your friend decrypt the cipher text, he or she should use his/her private key as follows.

openssl rsautl -decrypt -inkey private_key.pem -in ciphertext.txt -out plaintext.txt

Digital Signature of a Message:

When sending an encrypted message to someone, sometimes we need to prove that we are the person who sent the message. Just because we use RSA encryption as above, we cannot prove that we sent it. In order to prove the senders identity, we can use digital signatures. A digital signature is nothing other than just a Hash value of the original message encrypted using the private key of the sender. Since the private key of someone is always with that perform, a digital signature can be produced only by the owner of the sender's private key. We can digitally sign as follows using the RSA private key.

openssl dgst -sha256 -sign private_key.pem -out signature.txt plaintext.txt

Now, at the recipients side, he or she can verify the senders signature by decrypting the signature file using the senders public key (which is available) and then recalculate the Hash value of the original message. Following command illustrates that functionality.

openssl dgst -sha256 -verify public_key.pem -signature signature.txt plaintext.txt

There are so many things we can do with the OpenSSL library. It is up to you to explore it further.

Monday, April 24, 2017

Using Autopsy Tool for Forensic Disk Analysis

Autopsy is a web based GUI interface for the Sleuthkit forensic investigation tool kit. It can be used for interesting forensic analysis works on images acquired from storage devices. In this blog post, I'm writing the basics steps to install and use Autopsy tool to analyze a disk image.

Since the usage of a real disk image taken from somebody's storage device raises privacy concerns, I'm preparing custom disk images by manually creating filesystems on binary files. So, first of all, let's create our testing disk image.

Creating a custom disk image:

(1) Creating a file of 500MB. 

  dd if=/dev/zero of=./mydisk.img bs=1M count=500

(2) Format the file with the FAT32 file system.

    sudo mkfs.vfat -F 32 mydisk.img

(3) Mount the filesystem.

    mkdir mount-point
  sudo mount mydisk.img ./mount-point/

(4) Create a text file and add some sample content.

    sudo touch mount-point/readme.txt
  sudo vim mount-point/readme.txt

    "This is just a text file for testing purposes."

(5) Now, delete the file you created.

    sudo rm mount-point/readme.txt

(6) Unmount the filesystem.

    sudo umount mount-point/

Using Autopsy tool for disk analysis:

It's time to analyse the disk image using Autopsy tool which is the GUI frontend for the Sleuthkit.

(1) Install Autopsy tool together with Sleuthkit on a Linux machine.

    sudo apt-get update
  sudo apt-get install autopsy

(2) Start Autopsy with root previleges.

    sudo autopsy

(3) Now, we can access the web interface using the following URL.


(4) Create a new case, a new host and finally give path to the above disk image. Once you reach the end of the creation of everything, you can see a button called "Analyze" in order to analyze the disk image.

(5) In this interface, click on the button for "File Analysis". Then you can see the files of the disk image. We can see our deleted text file in red color.

(6) In the above screen, you can see that there is a column called "META". Click the item of the deleted "readme.txt" file under this "META" column. Now you will see some meta data of the file.

(7) In the above screen, note that the file size is 47 bytes. Since the sector size of this disk image is 512 bytes, this file just resides in a single sector. That is the sector shown as "2038" in the above screenshot. Click on that sector number to view it.

 (8) On this new screen, we can view the content of the file in different formats such as ASCII and Hex.

(9) Click on the "Export Contents" button to export the deleted file and save it somewhere in your local storage.
It just saves as a raw file without the proper file extesion.

(10) We can check the file type of the exported file using various ways.

    file vol1-Sector2038.raw

(11) Let's rename the file to the correct file extension type.

    mv vol1-Sector2038.raw textfile.txt

(12) Finally, take a look at the file contents to see that is the file we created.

    cat textfile.txt

There are so many features in Autopsy tool which we can explore.




Saturday, April 22, 2017

Taking a Linux RAM Image

For various purposes such as forensic investigatins and debugging of Linux systems, we need to have a RAM image taken from a running Linux system. While there are various ways to do it, I explored an easy and interesting way using a special kernel module for Linux called LiME. I will explain the steps one by one.

(1) Download LiME

(2) Go into the downloaded directory and compile the kernel module.


(3) Load the kernel module and save RAM dump to a file in one line.

sudo insmod lime-4.4.0-70-generic.ko "path=/home/asanka/Desktop/asanka-ram/mem.lime format=lime"

(4) If we want to take another RAM dump, first we have to unload the kernel module.

lsmod | grep lime
rmmod lime

(5) Now let's capture again but this time we use the 'raw' format.

sudo insmod lime-4.4.0-70-generic.ko "path=/home/asanka/Desktop/asanka-ram/mem.raw format=raw" 

(6) Analysis of the captured RAM image is a seperate topic. However, we can perform the most basic things with this RAM image as a start.

strings mem.raw | less
strings mem.raw | grep "key word"



Inspecting File Metadata using exiftool

Inspecting metadata of a file is a way to reveal so many interesting things about it. Specially, image files which are taken as pictures from cameras contains wonderful amount of information. There's a nice tool called exiftool which we can use for this purpose. I'll briefly write about how we can install it and use to inspect some files.

(1) In order to install exiftool, you can either use the package repositories or just download the source code of exiftool and build locally.

#To install using package repositories, issue to following command in the terminal.

sudo apt-get update
sudo apt-get install libimage-exiftool-perl 

#If you want to build exiftool from the source code, first you should download the code from the following place.

#Extract the downloaded compressed file and move into it.

tar xvzf Image-ExifTool-10.49.tar.gz Image-ExifTool-10.49/ 
cd Image-ExifTool-10.49/

#Compile and install the tool using the following commands.

perl Makefile.PL
make test
sudo make install

(2) It's time to inspect a file. Let's take a picture from a camera and then inspect it as follows.

exiftool file-name.jpg

The screen shot shown above illustrates the output of the exiftool ran against an image file.

Thursday, January 26, 2017

Using PDFTK to Process PDF Documents

When dealing with PDF documents, we come across different requirements such as splitting a document in to parts and sometimes merging multiple documents together. I recently came across a great tool which we can use to do various tasks with PDF documents. I decided to list down few things I can do using pdftk tool and leave a link to a better resource.

(1) Install pdftk tool.

sudo apt-get update

sudo apt-get install pdftk

(2) Suppose I want to create a new pdf file extracted from the contents of the page numbers 31 to 37 in a larger pdf file. We can do that as follows.

pdftk input.pdf cat 31-37 output output.pdf

(3) Merging two documents one at the end of another.

pdftk input-file1.pdf input-file2.pdf cat output new-document.pdf

(4) Selecting few pages from multiple documents and putting them together into a single document.

pdftk A=input-file1.pdf B=input-file2.pdf cat A110-117 B2-3 output new-document.pdf

That's it!


Wednesday, January 25, 2017

Independent Component Analysis (ICA)

When we want to separate two signals which are mixed up, one interesting method we can use is Independent Component Analysis (ICA).  I think I'm not knowledgeable enough to explain how this whole thing works but there are plenty of explanations about it in the web. Take a look at the references listed at the end for any further details. But, the purpose of this article is to put the codes I used recently for an ICA job so that I will not forget how to use it in the future.

In order to perform ICA on Python we need to install an important package first. Let's do it first.

sudo pip install --upgrade pip
pip install -U scikit-learn

Now, it's time to write the Python script. The following script is taking two wav files as input which contains two mixed signals in different ways. Then it generates another two new wav files which contains the separated signals.

Blind source separation using FastICA

An example of estimating sources from noisy data.

:ref:`ICA` is used to estimate sources given noisy measurements.
Imagine 3 instruments playing simultaneously and 3 microphones
recording the mixed signals. ICA is used to recover the sources
ie. what is played by each instrument. Importantly, PCA fails
at recovering our `instruments` since the related signals reflect
non-Gaussian processes.


import os
import wave
import pylab
import matplotlib

import numpy as np
import matplotlib.pyplot as plt
from scipy import signal
from import wavfile

from sklearn.decomposition import FastICA, PCA


matplotlib.rcParams['ps.useafm'] = True
matplotlib.rcParams['pdf.use14corefonts'] = True
matplotlib.rcParams['text.usetex'] = True

# read data from wav files
sample_rate1, samples1 ='100000010mix1.wav')
sample_rate2, samples2 ='100000010mix2.wav')

print 'sample_rate1', sample_rate1
print 'sample_rate2', sample_rate2

S = np.c_[samples1, samples2]

ica = FastICA(n_components=2)
S_ = ica.fit_transform(S)  # Reconstruct signals

print 'original signal=', S
print 'recovered signal=', S_
print 'extracted signal1', S_[:,0]
print 'extracted signal2', S_[:,1]

# write data to wav files
scaled1 = np.int16(S_[:,0]/np.max(np.abs(S_[:,0])) * 32767)
wavfile.write('extracted-signal-1.wav', sample_rate1, scaled1)

scaled2 = np.int16(S_[:,1]/np.max(np.abs(S_[:,1])) * 32767)
wavfile.write('extracted-signal-2.wav', sample_rate2, scaled2)

# Plot results

pylab.figure(num=None, figsize=(10, 10))

pylab.title('(received signal 1)')
pylab.xlabel('Time (s)')
pylab.ylabel('Sound amplitude')

pylab.title('(received signal 2)')
pylab.xlabel('Time (s)')
pylab.ylabel('Sound amplitude')

pylab.title('(extracted signal 1)')
pylab.xlabel('Time (s)')
pylab.ylabel('Sound amplitude')

pylab.title('(extracted signal 2)')
pylab.xlabel('Time (s)')
pylab.ylabel('Sound amplitude')


Run this Python script with the two wav files in the same directory and you will get the output signals as wav files in addition to seeing waveform plots of all these signals.