IFS

How many times have you written scripts and a had bad time with those having spaces? The remedy to this situation is your IFS value.

IFS or Internal Field Seperator holds the value which seperates the various entities. This can be file names, values read into a script by read etc. It is the character or characters designated as whitespace by the operating system.

The IFS is set to the newline and space character. The global variable $IFS stores the value. To view the exact value stored in IFS execute:

echo "$IFS" | cat -vTE
 ^I$
$

Running echo “$IFS” will not give you any visible output (after all, you are going to see a space and a newline). cat -vTE displays non printable characters , tabs as ^I and ends each line with a $ sign.

In a script which utilises filenames (with spaces), it is always preferable to change the IFS to include only the newline character opposed to the default space and newline character. Lets check out one such script which accepts filenames wih spaces. This scripts simply prints the file names in your current directory. (Remember to create some files in your currenct directory which has spaces. You may try the same script removing the lines with the IFS variable in reference to see the difference)

#!/bin/bash
OIFS=$IFS # Original IFS

IFS=$(echo -en "\n\b") # New IFS

for fil in $(ls -1 $PWD); do
	echo $fil
done

IFS=$OIFS # Restore earlier IFS

IFS can also be used to read files with lines sepearated by a special character. For example in the /etc/passwd, to store the various entries like username, homedirectory etc.

The following script uses the while construct to determine the users who have the shell portion as /bin/false

#!/bin/bash

OIFS=$IFS
IFS=':'

while read username password userid groupid comments homedir shell_avail
do
	if [[ $shell_avail == /bin/false ]]; then
                echo "$username has no shell"
        fi

done < /etc/passwd
IFS=$OIFS

In the above script each of the 7 portions of the /etc/passwd file is assigned to the 7 variables
username password userid groupid comments homedir shell_avail with the read command. The if portion in the script compares the seventh variable – shell_avail to /bin/false to determine the username and outputs it.

From now on you can use the IFS variable for all those files with spaces and extracting values separated by a special character.

Tweet This Post

Continue Reading

RootKits and anti rootkits

A rootkit is a collection of programs that enable an attacker to get the same privilage as the root user in a linux or unix system. The word is composed of two portions: ‘root’ – meaning the application will provide the highest access level of the root/administrator in the system and ‘kit’ – meaning it has a number of tools.

Attackers after getting access to a server, will install a rootkit to hide their identity and run desired scripts anywhere within the server. It makes the life of a hacker easy once installed. Rootkits are not easily detectable. Sometimes, if the rootkit is one of the latest ones without a diagnosis, the server will have to be rebuild from scratch.

A rootkit will have multiple applications for cracking the entire server, some of them are:

Server Access Applications (Back door application)
These applications will create a backdoor to log in to the hacked system without using the exploit again.

Log clearing Applications
These applications clear the logs of the events performed by the hacker or the applications used. They all the associated log files in the server.

Packet sniffing Applications
These applications monitor the data through the various interfaces in the server at particular ports.

Malicious Scripts
Many scripts will be installed like IRC bots, ddos daemons, spam servers, trojans, worms etc.

There are mainly two kinds of root kits. The application rootkit and the kernel rootkit.

Application rootkits
These rootkits mimic a particular application and will hide the attackers files/processes from being revealed by the original application. To illustrate, a rootkit ls application will perform all the task of a normal ls but will not display any of the files of the attacker. Other application rootkits will create backdoors for unauthorised access, packet sniffers etc which go undetected or are hidden by renaming. Application rootkits are the most common.

Kernel rootkits
Kernel rootkits modify the kernel and apply patches to the kernel and device drivers. They also hide the applications and files of the attacker. As antivirus and other applications run beneath the kernel, they are the most undetectable rootkits.

‘Prevention is better than cure’ – as this saying goes, it is always better to keep the system secure and updated when ever possible to stop these installations. There are some applications which help detect any known rootkits running in the system. One such is the chkrootkit.

chkrootkit is one of the popular rootkit detectors (an anti-rootkit) and it is know to detect common rootkits on unix/linux servers. chkrootkit relies on basic string processing techniques to determine the presence of rootkits. It scans specific sytem files and binaries targeted by rootkits for known signatures.

The following are the instructions to install chkrootkit version 0.49 in a server.

cd /usr/local/

wget ftp://ftp.pangeia.com.br/pub/seg/pac/chkrootkit.tar.gz

wget ftp://ftp.pangeia.com.br/pub/seg/pac/chkrootkit.md5

md5sum -c chkrootkit.md5 # to check if the downloaded file is intact

tar -xzf chkrootkit.tar.gz

cd chkrootkit-0.49/

make sense

./chkrootkit

chkroootkit will check all the files and display the status of the files analysed. This information may be logged for future reference. For this a cron job may be setup to be run at least once a month.

Inserting an entry like the one below into the systems cron tasks (executed atleast once a month) will send the report of the chkrootkit vulnerabilities to the administrator conserned.

/usr/local/chkrootkit-0.49/chkrootkit | mail -s "chkrootkit report $(date +%d/%m/%y)" "admin@domain.com"

Tweet This Post

Continue Reading

PostgreSQL for the sage – Must know basics for the system administrators

PostgreSQL or Postgres is an object-relational database management system (ORDBMS). Unlike MySQL, PostgreSQL is not controlled by any single company, it is a community developed project. It is a advanced version of the ‘Ingres’ Database project (which is how the project gets the name post-ingres or postgres ).

Postgres is one of the best open-source database alternative which is fully object oriented and transactions compliant. It has stored procedures, multiple views and a huge set of datatypes. Some of the other notable features are as follows.

Objects and Inheritance

Database consists of objects and the database administrators can design custom or user-defined objects for the tables. Inheritance is another feature. Tables can be set to inherit their characteristics from a “parent” table.

Functions

Functions can be used in Postgres. These can be written in the postgres’ own procedural language called ‘PL/pgSQL’ which resembles Oracle’s procedural language ‘PL/SQL’ or any other common scripting languages which support posgtres’ procedural language like PL/Perl, plPHP, PL/Python, PL/Ruby etc. Run the following in the psql client to determine if functions is enabled:

SELECT true FROM pg_catalog.pg_language WHERE lanname = 'plpgsql'; 

To create user-defined functions we use the CREATE OR REPLACE FUNCTION command. Example:

CREATE OR REPLACE FUNCTION fib (

fib_for integer

) RETURNS integer AS $$

BEGIN

IF fib_for < 2 THEN

RETURN fib_for;

END IF;

RETURN fib(fib_for - 2) + fib(fib_for - 1);

END;

$$ LANGUAGE plpgsql;

Indexes

An index is like a summary of a certain portion of the table. It is an optimization technique which increases speed of accessing records from a database. PostgreSQL supports indexes like Btree, hash etc. User-defined index methods can also be created. Indexes are created on tables with respect to a particular field (based on which there are a number of queries). As an example for a table:

CREATE TABLE name (

id integer,

fname varchar

lname varchar

);

To create an index on table name with respective to the field id (as there are many queries on this table requesting for firstname or lastname from the id provided), we use the index:

CREATE INDEX name_id_index ON name (id);

Triggers

Triggers are events or functions run upon the action of certain SQL statements which modify data in some records. Depending on the kind of modification we can have multiple triggers in a database. Postgres supports multiple triggers written in PL/PgSQL or it’s scripting counterparts like PL/Python. The trigger function must be defined before the trigger can be created. The trigger function must be declared as a function taking no arguments and returning type trigger. CREATE TRIGGER command is used to declare triggers.

Concurrency

PostgreSQL ensures concurrency with the help of MVCC (Multi-Version Concurrency Control), which gives the database user a “snapshot” of the database, allowing changes to be made without being visible to other users until a transaction is committed.

PostgreSQL’s MVCC keeps all of the versions of the data together in the same partition in the same table. By identifying which rows were added by which transactions, which rows were deleted by which transactions, and which transactions have actually committed, it becomes a straightforward check to see which rows are visible for which transactions.

Inorder to accomplish this, Rows of a table are stored in PostgreSQL as a tuple. Two fields of each tuple are xmin and xmax. Xmin is the transaction ID of the transaction that created the tuple. Xmax is the transaction ID of the transaction that deleted it (if any).

Along with the tuples in each table, a record of each transaction and its current state (in progress, committed, aborted) is kept in a universal transaction log.

When data in a table is selected, only those rows that are created and not destroyed are seen. That is, each row’s xmin is observed. If the xmin is a transaction that is in progress or aborted, then the row is invisible. If the xmin is a transaction that has committed, then the xmax is observed. If the xmax is a transaction that is in progress or aborted and not the current transaction, or if there is no xmax at all, then the row is seen. Otherwise, the row is considered as already deleted.

Insertions are straightforward. The transaction that inserts the tuple simply creates it with the xmax blank and the xmin set to its transaction ID. Deletions are also straightforward. The tuple’s xmax is set to the current transaction. Updates are no more than a concurrent insert and delete.

Views

A view is a table which does not exist in the database. It is a virtual table created from fields in various tables and is joined together based on some criteria. Views can be used in place of tables and will accomplish the task same as that of a table. The CREATE VIEW statement is used to accomplish this eg:

CREATE VIEW best_sellers AS

SELECT * FROM publishers WHERE demand LIKE 'high';

Foreign Keys

The primary key used in one table which is used to refer to the records in a second table is called the foreign key of the second table.

CREATE TABLE products (
    product_no integer PRIMARY KEY,
    name text,
    price numeric
);
CREATE TABLE orders (
    order_id integer PRIMARY KEY,
    product_no integer REFERENCES products (product_no),
    quantity integer
);

Here product_no is the foreign key in the second table created. The foreign key field may have values which are repeated unlike primary keys.

Files Users and Configuration

The main configuration file of Postgres is postgresql.conf. This can be located in the ‘data’ directory. It may be present either in /var/lib (/var/lib/pgsql/data/postgresql.conf) or /usr/local (/usr/local/pgsql/data/postgresql.conf). Temporary changes to the configurations can be made using postmaster command.

The init script that starts the postgres service is /etc/init.d/postgresql . It runs a number of child processes concurrently. The postgres server process is postmaster. These processes and files associated with PosgreSQL are owned by the user/group postgres. The default port used for database connections is 5432

The user postgres is the PostgreSQL database superuser. We can create a number of super users for the database (this accomplished by the create role command ), however, the default super user is postgres. The postgres user has the privilege to access all the databases and files in the server (Unless the user root is created in postgres as a superuser).

Client Authentication is controlled by the file pg_hba.conf in the data directory, e.g., /var/lib/pgsql/data/pg_hba.conf. (HBA stands for host-based authentication.)

Each record specifies a connection type, a client IP address range (if relevant for the connection type), a database name or names, and the authentication method to be used for connections matching these parameters.A record is typically in one of two forms:

local database authentication-method [ authentication-option ]

host database IP-address IP-mask authentication-method [ authentication-option ]

local : This record pertains to connection attempts over Unix domain sockets.

host : This record pertains to connection attempts over TCP/IP networks.

database : Specifies the database that this record applies to. The value all specifies that it applies to all databases, while the value sameuser identifies the database with the same name as the connecting user.

authentication methods

trust: The connection is allowed unconditionally.

reject: The connection is rejected unconditionally.

password: The client is required to supply a password which is required to match the database password that was set up for the user.

md5: Like the password method, but the password is sent over the wire encrypted using a simple challenge-response protocol.

ident: This method uses the “Identification Protocol” as described in RFC 1413. It may be used to authenticate TCP/IP or Unix domain socket connections, but its reccomended use is for local connections only and not remote connections.

Front-ends

The minimalistic front-end for PostgreSQL is the psql command-line. It can be used to enter SQL queries directly, or execute them from a file. phpPgAdmin is a web-portal used for PostgreSQL administration written in PHP and based on the popular phpMyAdmin. Likewise pgAdmin is a graphical front-end administration tool for PostgreSQL, which has support on multiple platforms. The latest stable version of the same is pgAdmin III.

Some administration related commands

Command to login to psql database mydb as user myuser:

psql -d mydb -U myuser

Command to login to psql database mydb as user myuser on a different host myhost:

psql -h myhost -d mydb -U myuser

If the port the server runs is different we use -p [port number] . Upon entering the psql shell the prompt will show the database name currently being used. In the above example it will show

mydb=> (if logged in as an ordinary user )

mydb=# (if logged in as a super user like postgres)

Create a PostgreSQL user

There are two ways to create a postgres database user. The only user initially allowed to create users is postgres. So one has to switch to this user before creating other users with varying privileges.

1. Creating the user in the shell prompt, with createuser command.

switch to the postgres user with:

su - postgres

createuser tom

Shall the new role be a superuser? (y/n) n

Shall the new role be allowed to create databases? (y/n) y

Shall the new role be allowed to create more new roles? (y/n) n

2. Creating the user in the PSQL prompt, with CREATE USER command.

switch to the postgres user with:

su - postgres

create user mary with password 'marypass';

Creating and deleting a PostgreSQL Database

There are two way to create databases.

1. Creating database in the PSQL prompt, with createuser command.

CREATE DATABASE db1 WITH OWNER tom;

2. Creating database in the shell prompt, with createdb command.

createdb db2 -O mary

To delete an entire database from within the psql prompt do :

DROP DATABASE db1;

Determining execution time of a query

Turn on timing with

\timing

Now execute the qery:

SELECT * from db1.employees ;

Time: 0.065 ms

Calculate postgreSQL database size in disk

SELECT pg_database_size('db1');

to get the values in human readable format

SELECT pg_size_pretty(pg_database_size('db1'));

to calculate postgreSQL table size in disk

SELECT pg_size_pretty(pg_total_relation_size(‘big_table’));

Slash commands used in psql

To list all slash commands and thier purpose. Login to psql and issue to the command \? . Some of the most commonly used slash commands are the following:

List databases	\l
System tables	\dS
Types	\dT
Functions	\df
Operators	\do
Aggregates	\da
Users	\du
Quit from psql	\q
Connect to different database db2	\c db2
Describe Table/index/view/sequence	\d

The below can be used with a specific table/index/view name for description of the specific table/index/view

Tables	\dt
Indexes	\di
Sequences	\ds
Views	\dv

Useful Bash commands

Bash command to list all the postgresql databases:

psql -l #This can be run as a unix user who is also a super user in postgresql

Indirect bash command to list all the postgresl users:

psql -c '\du' #-c is used to run an internal or sql command in psql shell

Backing up and restoring databases

To dump the database to an sql file use the bash command:

pg_dump mydb > db.out

To restore a database from an sql backup file (via bash)

psql -d newdb -f backupdb.out

or

psql -f backupdb.out newdb

(here the database newdb must be already created and the file backupdb.out must be present in the current directory)

To take the backup of all the Postgres databases in the server:

pg_dumpall > /var/lib/pgsql/backups/dumpall.sql

(Only possible with the postgres or the database superuser )

Resetting database user’s password

To change the password for a database user (say ‘thomas’):

ALTER USER thomas WITH PASSWORD 'newpassword';

This same command can be used to reset the password for the postgresql super user postgres, but in this case, you will have to enable password less login for postgres user by adding the following line to the top of the file pg_hba.conf in the data directory of postgres. Once the password is reset this line can be removed:

local	all	postgres	trust

Next we issue the same command but for the user postgres

ALTER USER postgres WITH PASSWORD 'newpassword';

To create a super user via bash with multiple roles

createuser -sPE mysuperuser

Instead of this we can also use the below psql shell command:

CREATE ROLE mysuperuser2 WITH SUPERUSER CREATEDB CREATEROLE LOGIN ENCRYPTED PASSWORD 'mysuperpass2';

Physical database files in postgres

The files in data/base are named by the oid (Object Identifier) of the database record in

pg_database, like this:

cd /var/lib/pgsql/data/base

ls -l

total 33

drwx------ 22 postgres postgres 4096 Jul 23 20:06 ./

drwx------ 11 postgres postgres 4096 Aug  1 05:59 ../

drwx------  2 postgres postgres 4096 Jun 20 09:32 1/

drwx------  2 postgres postgres 4096 Mar  3 13:36 10792/

drwx------  2 postgres postgres 4096 Jun 20 15:09 10793/

drwx------  2 postgres postgres 4096 May 27 01:40 16497/

drwx------  2 postgres postgres 4096 May 27 01:40 16589/

drwx------  2 postgres postgres 4096 Jun 20 10:28 16702/

drwx------  2 postgres postgres 4096 May 27 01:40 16764/

drwx------  2 postgres postgres 4096 May 27 01:40 16785/

drwx------  2 postgres postgres 4096 Aug  1 04:37 16786/

drwx------  2 postgres postgres 4096 Aug  1 04:36 19992/

drwx------  2 postgres postgres 4096 May 27 01:40 19997/

To obtain the oid, execute the following command in psql prompt

postgres=# select oid,datname from pg_database order by oid;

   oid  |         datname

---------+--------------------------

1 | template1

10792 | template0

10793 | postgres

16497 | gadgetwi_Unable

16589 | vimusicc_filehost

16702 | personea_altissimo

16764 | shopping_businessfinance

16785 | ansonyi_wp2

16786 | ansonyi_wp

19992 | globook_PostgreSQL

Tweet This Post

Continue Reading

All about SSL

SSL or Secure Sockets Layer (Notice the last s in sockets) is used to secure the communication over the internet. This technique was introduced by Netscape. It uses the RSA public key cryptography for encryption/decryption.

In the protocol stack used in the internet. The SSL protocol runs above TCP/IP and below higher-level protocols such as HTTP or IMAP.

The SSL protocol includes two sub-protocols:
1) SSL record protocol
2) SSL handshake protocol

The SSL record protocol defines the format used to transmit data. The SSL handshake protocol involves using the SSL record protocol to exchange a series of messages between an SSL-enabled server and an SSL-enabled client when they first establish an SSL connection.

Now SSL for the layman
SSL basically creates an encrypted communication channel between the two parties involved in the communication. For a third person involved in the middle of this communication channel, the data seems to be garbled. (Read the rest of this entry…)

Tweet This Post

Continue Reading

mod_php explained

Preface

Like in the case of all apache modules, you can either compile PHP as a static module or compile it as a dynamic module.  In the case of static module , you  can’t perform any modification for the module without recompiling the binary to which it is attached.  For eg.  you can’t add ssl support for the mod_php without re-compiling apache as a whole. And any failure in the compilation may cause downtime for the entire webserver also, including plain html support.

The advantage is that it provides a faster  performance, because the module is initialized  whenever the apache binary is started.

In the case of a dynamic mod_php installation, the necessary modifications or module additions can be done by recompiling the module alone. There is no need to recompile the Apache as the mod_php is not linked with the binary of apache.  But since the webserver loads the module on the fly, it needs to load, initialize and then execute the module.  So it can create some level of slowness while processing php pages.

How does mod_php work?

When PHP  is loaded into Apache as a module (using mod_php), each Apache process will contain an instance of mod_php or PHP interpreter also.   The interpreter comes with a bundle of libraries we enabled during compilation and each  process can make use of these libraries to process the requests. This means that the Apache process that just started  to load a simple HTML page  too will contain a PHP interpreter with all assigned libraries which inturn means resource consumption.

When the webserver gets an HTTP request. The  request header  contains the path to the requested document

e.g. access.log:    xx.xx.xx.xx – - [22/June/2010:21:14:53 -0700] “GET /info.php?=PHPE9568F35-D428-11d2-A769-00AA001ACF42 HTTP/1.1″ 200 2146 “http://domain.com/info.php” “Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9.1.7) Gecko/20100106 Ubuntu/9.10 (karmic) Firefox/3.5.7″

1. The request will be redirected to the document root of the domain and then to the file “info.php” , if it fails then the corresponding error message will be given.

2. The info.php file is to be processed. It follows the following steps

Normally every httpd.conf file will have an entry like this

AddType application/x-httpd-php .php5 .php4 .php

It instructs the webserver that the files with extension .php ,.php4,.ph5 are of PHP mime type . The TypesConfig directive sets the location of the MIME types configuration file. This file controls what Internet media types are sent to the client for  given file extension(s).  Sending the correct media type to the client  is important so they know how to handle the content of the file.

root@new [/usr/local/apache/conf]# cat mime.types | grep x-httpd-php
application/x-httpd-php-source          phps
application/x-httpd-php          php php3 php4 php5 php6
root@new [/usr/local/apache/conf]#

Here we can see that the Mime type to be used for files with extension .php  is application/x-httpd-php , while the file with .phps is to be mapped to the php mime type application/x-httpd-php-source .

The webserver identifies that the requested file is of Mime type x-httpd-php.

To handle or process it , the apache has to load the corresponding module. Since it is a php type , the module mod_php will be loaded and it will execute the file.

Since apache is a HTTP server. It gets the HTTP requests and answers with the HTML code.  So the mod_php will execute the commands within php flag and creates the HTML page dynamically and send it back to the client – internet browser which sent HTTP request.

Security concerns / Implications

You can see that  every request or execution of a php file through web is initiated by the webserver. So the webserver acts as the parent of every php execution through web. It imposes a great security threat. Since apache is being executed as an apache  user, all process will be owned by that user. By default  it is “nobody” or “apache”.  Let me try to explain.

If your  web application performs some operations in the db, unless that database (eg: a flat text DB) has built-in access control, you will have to make the database accessible to the “nobody” user. This means a malicious script could access and modify the database, even without a username and password.  Such can be the case with various configuration files too.  Unless you protect these directories or applications with necessary authorization techniques like .htaccess, session control etc. There is a high possibility of attack through webapplication.

Another dangerous issue is of root escalation. If the webserver has a bug, by exploiting that bug, a malicious user can gain some root privileges or escalated to root. Its quite alarming situation as an escalated apache user can do any sort of  actions without any level of authentication.

Also it is difficult to identify the script which performs the malicious activity as all php scripts will be executed as “nobody”

Since PHP applications are executed as web server user, you need to give access and write permissions for the directories wherever the application  is supposed to be working. Sometimes you may be forced to give 777 permissions and it invites lot of attacks.

The files created by php applications will be owned by user “nobody” . So the user will not be able to delete the files unless it is done through another php application. Otherwise he needs to contact the server admin to get the same.

As a security measure, we may be forced to block mails from “nobody” users . But it can create mails generated from php applications being blocked in the server. Various php applications widely used for spamming . So some servers are configured to block mails from nobody users. This creates inconvenience to users.

Tweet This Post

Continue Reading