How to create a RPM from source package(Creating CSF RPM)

We create RPM’s from the Source for a package. As an initial step, compile and install the source using normal procedures just to confirm all necessary libraries/dependencies are met.

Here I am explaining the whole process to create an RPM for CSF that could be  installed on cPanel servers.

Getting ready with the Source Files:

Download Latest CSF from here : http://www.configserver.com/free/csf.tgz

Extract the tarball.

[root@server new]# tar -xvf csf.tgz
[root@server new]# ls
csf  csf.tgz

Install to check all libraries/dependencies are met.

[root@server csf]# cd csf
[root@server csf]# sh install.cpanel.sh
*WARNING* TESTING mode is enabled - do not forget to disable it in the configuration
Installation Completed

We confirmed all libraries/dependencies are met for installing CSF.  We need to rename the tarball to define a version so that it could be used in the SPEC file.

[root@server csf]# cd ..
[root@server new]# ls
csf  csf.tgz
[root@server new]#mv csf csf-0.0.1
[root@server new]# tar -cvf csf-0.0.1.tar.gz csf-0.0.1

0.0.1 defines the version. (Can assign any). Now we are ready with the source file for CSF csf-0.0.1.tar.gz that could be used to generate RPM.

Building the RPM:

Install RPMBUILD tool:

[root@server ~]# yum install rpm-build

We never build RPM’s as ‘root’ users, but as normal users, because root can alter any file on the system, it was easy to inadvertently alter a running system by adding extraneous files or removing important files during interim builds of an RPM. Earlier RPM’s were build as root user but recently the RPM system changed to allow any user to build RPMs in a home directory. Building an RPM without the privileges of root prevents changes to core system files and hence we are on the safer side.

Initial Set-up:

We need to create a directory hierarchy for the rpm build tool to work with. To begin with, create a directory under your home directory which will act as the root directory for the build process. Lets make a directory ‘csfrpm’ under home directory for this purpose.

[sage@server ~]$ mkdir -p /home/your_username/csfrpm

[sage@server ~]$ cd /home/your_username/csfrpm

Create Five sub-directories under csfrpm.

[sage@server csfrpm]$ mkdir BUILD RPMS SOURCES SPECS SRPMS

• BUILD: BUILD is used as a space to compile the software.
• RPMS: RPMS contains the binary RPM that rpmbuild builds.
• SOURCES: SOURCES is for source code.
• SPECS: SPECS contains your spec file or files—one spec file per RPM you want to build.
• SRPMS: SRPMS contains the source RPM built during the process.

[sage@server csfrpm]$ ls
BUILD  RPMS  SOURCES  SPECS  SRPMS

Copy the source code that we have created(csf-0.0.1.tar.gz) to the SOURCES folder. Make sure that the owner  for source file is your_username.

[sage@server ~]$ cp csf-0.0.1.tar.gz /home/your_username/csfrpm/SOURCES/

Create the SPEC file:

SPEC file is noting but the configuration for rpmbuild tool.

[sage@server csfrpm]$ vi SPECS/csf.spec

# This is a sample spec file for csf

%define _topdir         /home/your_username/csfrpm
%define name            csf
%define release         0
%define version         0.0.1
BuildRoot: %{_tmppath}/%{name}-%{version}-root
Summary:                GNU csf
License:                GPL
Name:                   %{name}
Version:                %{version}
Release:                %{release}
Source:                 %{name}-%{version}.tar.gz
Group:                  Security/Tools

%description
A Stateful Packet Inspection (SPI) firewall, Login/Intrusion Detection and Security application for Linux servers.

%prep
%setup -q

%install
./install.cpanel.sh prefix=$RPM_BUILD_ROOT/usr/local

%clean
%{__rm} -rf %{buildroot}

%files
%defattr(-,root,root,0755)

Finally Build the RPM:

[sage@server csfrpm]$ rpmbuild -v -bb --clean SPECS/csf.spec

You can see the result if everything went fine:

Processing files: csf-0.0.1-0
Checking for unpackaged file(s): /usr/lib/rpm/check-files /var/tmp/csf-0.0.1-root
Wrote: /home/your_username/csfrpm/RPMS/i386/csf-0.0.1-0.i386.rpm
Executing(%clean): /bin/sh -e /var/tmp/rpm-tmp.84469
+ umask 022
+ cd /home/your_username/csfrpm/BUILD
+ cd csf-0.0.1
+ /bin/rm -rf /var/tmp/csf-0.0.1-root
+ exit 0
Executing(--clean): /bin/sh -e /var/tmp/rpm-tmp.84469
+ umask 022
+ cd /home/yuor_username/csfrpm/BUILD
+ rm -rf csf-0.0.1
+ exit 0

The RPM is written to /home/your_username/csfrpm/RPMS/i386/csf-0.0.1-0.i386.rpm

Tweet This Post

Continue Reading

PostgreSQL for the sage – Must know basics for the system administrators

PostgreSQL or Postgres is an object-relational database management system (ORDBMS). Unlike MySQL, PostgreSQL is not controlled by any single company, it is a community developed project. It is a advanced version of the ‘Ingres’ Database project (which is how the project gets the name post-ingres or postgres ).

Postgres is one of the best open-source database alternative which is fully object oriented and transactions compliant. It has stored procedures, multiple views and a huge set of datatypes. Some of the other notable features are as follows.

Objects and Inheritance

Database consists of objects and the database administrators can design custom or user-defined objects for the tables. Inheritance is another feature. Tables can be set to inherit their characteristics from a “parent” table.

Functions

Functions can be used in Postgres. These can be written in the postgres’ own procedural language called ‘PL/pgSQL’ which resembles Oracle’s procedural language ‘PL/SQL’ or any other common scripting languages which support posgtres’ procedural language like PL/Perl, plPHP, PL/Python, PL/Ruby etc. Run the following in the psql client to determine if functions is enabled:

SELECT true FROM pg_catalog.pg_language WHERE lanname = 'plpgsql'; 

To create user-defined functions we use the CREATE OR REPLACE FUNCTION command. Example:

CREATE OR REPLACE FUNCTION fib (

fib_for integer

) RETURNS integer AS $$

BEGIN

IF fib_for < 2 THEN

RETURN fib_for;

END IF;

RETURN fib(fib_for - 2) + fib(fib_for - 1);

END;

$$ LANGUAGE plpgsql;

Indexes

An index is like a summary of a certain portion of the table. It is an optimization technique which increases speed of accessing records from a database. PostgreSQL supports indexes like Btree, hash etc. User-defined index methods can also be created. Indexes are created on tables with respect to a particular field (based on which there are a number of queries). As an example for a table:

CREATE TABLE name (

id integer,

fname varchar

lname varchar

);

To create an index on table name with respective to the field id (as there are many queries on this table requesting for firstname or lastname from the id provided), we use the index:

CREATE INDEX name_id_index ON name (id);

Triggers

Triggers are events or functions run upon the action of certain SQL statements which modify data in some records. Depending on the kind of modification we can have multiple triggers in a database. Postgres supports multiple triggers written in PL/PgSQL or it’s scripting counterparts like PL/Python. The trigger function must be defined before the trigger can be created. The trigger function must be declared as a function taking no arguments and returning type trigger. CREATE TRIGGER command is used to declare triggers.

Concurrency

PostgreSQL ensures concurrency with the help of MVCC (Multi-Version Concurrency Control), which gives the database user a “snapshot” of the database, allowing changes to be made without being visible to other users until a transaction is committed.

PostgreSQL’s MVCC keeps all of the versions of the data together in the same partition in the same table. By identifying which rows were added by which transactions, which rows were deleted by which transactions, and which transactions have actually committed, it becomes a straightforward check to see which rows are visible for which transactions.

Inorder to accomplish this, Rows of a table are stored in PostgreSQL as a tuple. Two fields of each tuple are xmin and xmax. Xmin is the transaction ID of the transaction that created the tuple. Xmax is the transaction ID of the transaction that deleted it (if any).

Along with the tuples in each table, a record of each transaction and its current state (in progress, committed, aborted) is kept in a universal transaction log.

When data in a table is selected, only those rows that are created and not destroyed are seen. That is, each row’s xmin is observed. If the xmin is a transaction that is in progress or aborted, then the row is invisible. If the xmin is a transaction that has committed, then the xmax is observed. If the xmax is a transaction that is in progress or aborted and not the current transaction, or if there is no xmax at all, then the row is seen. Otherwise, the row is considered as already deleted.

Insertions are straightforward. The transaction that inserts the tuple simply creates it with the xmax blank and the xmin set to its transaction ID. Deletions are also straightforward. The tuple’s xmax is set to the current transaction. Updates are no more than a concurrent insert and delete.

Views

A view is a table which does not exist in the database. It is a virtual table created from fields in various tables and is joined together based on some criteria. Views can be used in place of tables and will accomplish the task same as that of a table. The CREATE VIEW statement is used to accomplish this eg:

CREATE VIEW best_sellers AS

SELECT * FROM publishers WHERE demand LIKE 'high';

Foreign Keys

The primary key used in one table which is used to refer to the records in a second table is called the foreign key of the second table.

CREATE TABLE products (
    product_no integer PRIMARY KEY,
    name text,
    price numeric
);
CREATE TABLE orders (
    order_id integer PRIMARY KEY,
    product_no integer REFERENCES products (product_no),
    quantity integer
);

Here product_no is the foreign key in the second table created. The foreign key field may have values which are repeated unlike primary keys.

Files Users and Configuration

The main configuration file of Postgres is postgresql.conf. This can be located in the ‘data’ directory. It may be present either in /var/lib (/var/lib/pgsql/data/postgresql.conf) or /usr/local (/usr/local/pgsql/data/postgresql.conf). Temporary changes to the configurations can be made using postmaster command.

The init script that starts the postgres service is /etc/init.d/postgresql . It runs a number of child processes concurrently. The postgres server process is postmaster. These processes and files associated with PosgreSQL are owned by the user/group postgres. The default port used for database connections is 5432

The user postgres is the PostgreSQL database superuser. We can create a number of super users for the database (this accomplished by the create role command ), however, the default super user is postgres. The postgres user has the privilege to access all the databases and files in the server (Unless the user root is created in postgres as a superuser).

Client Authentication is controlled by the file pg_hba.conf in the data directory, e.g., /var/lib/pgsql/data/pg_hba.conf. (HBA stands for host-based authentication.)

Each record specifies a connection type, a client IP address range (if relevant for the connection type), a database name or names, and the authentication method to be used for connections matching these parameters.A record is typically in one of two forms:

local database authentication-method [ authentication-option ]

host database IP-address IP-mask authentication-method [ authentication-option ]

local : This record pertains to connection attempts over Unix domain sockets.

host : This record pertains to connection attempts over TCP/IP networks.

database : Specifies the database that this record applies to. The value all specifies that it applies to all databases, while the value sameuser identifies the database with the same name as the connecting user.

authentication methods

trust: The connection is allowed unconditionally.

reject: The connection is rejected unconditionally.

password: The client is required to supply a password which is required to match the database password that was set up for the user.

md5: Like the password method, but the password is sent over the wire encrypted using a simple challenge-response protocol.

ident: This method uses the “Identification Protocol” as described in RFC 1413. It may be used to authenticate TCP/IP or Unix domain socket connections, but its reccomended use is for local connections only and not remote connections.

Front-ends

The minimalistic front-end for PostgreSQL is the psql command-line. It can be used to enter SQL queries directly, or execute them from a file. phpPgAdmin is a web-portal used for PostgreSQL administration written in PHP and based on the popular phpMyAdmin. Likewise pgAdmin is a graphical front-end administration tool for PostgreSQL, which has support on multiple platforms. The latest stable version of the same is pgAdmin III.

Some administration related commands

Command to login to psql database mydb as user myuser:

psql -d mydb -U myuser

Command to login to psql database mydb as user myuser on a different host myhost:

psql -h myhost -d mydb -U myuser

If the port the server runs is different we use -p [port number] . Upon entering the psql shell the prompt will show the database name currently being used. In the above example it will show

mydb=> (if logged in as an ordinary user )

mydb=# (if logged in as a super user like postgres)

Create a PostgreSQL user

There are two ways to create a postgres database user. The only user initially allowed to create users is postgres. So one has to switch to this user before creating other users with varying privileges.

1. Creating the user in the shell prompt, with createuser command.

switch to the postgres user with:

su - postgres

createuser tom

Shall the new role be a superuser? (y/n) n

Shall the new role be allowed to create databases? (y/n) y

Shall the new role be allowed to create more new roles? (y/n) n

2. Creating the user in the PSQL prompt, with CREATE USER command.

switch to the postgres user with:

su - postgres

create user mary with password 'marypass';

Creating and deleting a PostgreSQL Database

There are two way to create databases.

1. Creating database in the PSQL prompt, with createuser command.

CREATE DATABASE db1 WITH OWNER tom;

2. Creating database in the shell prompt, with createdb command.

createdb db2 -O mary

To delete an entire database from within the psql prompt do :

DROP DATABASE db1;

Determining execution time of a query

Turn on timing with

\timing

Now execute the qery:

SELECT * from db1.employees ;

Time: 0.065 ms

Calculate postgreSQL database size in disk

SELECT pg_database_size('db1');

to get the values in human readable format

SELECT pg_size_pretty(pg_database_size('db1'));

to calculate postgreSQL table size in disk

SELECT pg_size_pretty(pg_total_relation_size(‘big_table’));

Slash commands used in psql

To list all slash commands and thier purpose. Login to psql and issue to the command \? . Some of the most commonly used slash commands are the following:

List databases	\l
System tables	\dS
Types	\dT
Functions	\df
Operators	\do
Aggregates	\da
Users	\du
Quit from psql	\q
Connect to different database db2	\c db2
Describe Table/index/view/sequence	\d

The below can be used with a specific table/index/view name for description of the specific table/index/view

Tables	\dt
Indexes	\di
Sequences	\ds
Views	\dv

Useful Bash commands

Bash command to list all the postgresql databases:

psql -l #This can be run as a unix user who is also a super user in postgresql

Indirect bash command to list all the postgresl users:

psql -c '\du' #-c is used to run an internal or sql command in psql shell

Backing up and restoring databases

To dump the database to an sql file use the bash command:

pg_dump mydb > db.out

To restore a database from an sql backup file (via bash)

psql -d newdb -f backupdb.out

or

psql -f backupdb.out newdb

(here the database newdb must be already created and the file backupdb.out must be present in the current directory)

To take the backup of all the Postgres databases in the server:

pg_dumpall > /var/lib/pgsql/backups/dumpall.sql

(Only possible with the postgres or the database superuser )

Resetting database user’s password

To change the password for a database user (say ‘thomas’):

ALTER USER thomas WITH PASSWORD 'newpassword';

This same command can be used to reset the password for the postgresql super user postgres, but in this case, you will have to enable password less login for postgres user by adding the following line to the top of the file pg_hba.conf in the data directory of postgres. Once the password is reset this line can be removed:

local	all	postgres	trust

Next we issue the same command but for the user postgres

ALTER USER postgres WITH PASSWORD 'newpassword';

To create a super user via bash with multiple roles

createuser -sPE mysuperuser

Instead of this we can also use the below psql shell command:

CREATE ROLE mysuperuser2 WITH SUPERUSER CREATEDB CREATEROLE LOGIN ENCRYPTED PASSWORD 'mysuperpass2';

Physical database files in postgres

The files in data/base are named by the oid (Object Identifier) of the database record in

pg_database, like this:

cd /var/lib/pgsql/data/base

ls -l

total 33

drwx------ 22 postgres postgres 4096 Jul 23 20:06 ./

drwx------ 11 postgres postgres 4096 Aug  1 05:59 ../

drwx------  2 postgres postgres 4096 Jun 20 09:32 1/

drwx------  2 postgres postgres 4096 Mar  3 13:36 10792/

drwx------  2 postgres postgres 4096 Jun 20 15:09 10793/

drwx------  2 postgres postgres 4096 May 27 01:40 16497/

drwx------  2 postgres postgres 4096 May 27 01:40 16589/

drwx------  2 postgres postgres 4096 Jun 20 10:28 16702/

drwx------  2 postgres postgres 4096 May 27 01:40 16764/

drwx------  2 postgres postgres 4096 May 27 01:40 16785/

drwx------  2 postgres postgres 4096 Aug  1 04:37 16786/

drwx------  2 postgres postgres 4096 Aug  1 04:36 19992/

drwx------  2 postgres postgres 4096 May 27 01:40 19997/

To obtain the oid, execute the following command in psql prompt

postgres=# select oid,datname from pg_database order by oid;

   oid  |         datname

---------+--------------------------

1 | template1

10792 | template0

10793 | postgres

16497 | gadgetwi_Unable

16589 | vimusicc_filehost

16702 | personea_altissimo

16764 | shopping_businessfinance

16785 | ansonyi_wp2

16786 | ansonyi_wp

19992 | globook_PostgreSQL

Tweet This Post

Continue Reading

Prevent your mail/IP from getting marked as SPAM/Blacklisted, A few TIPS!

“SPAM“! A word familiar to all Netizens.

Emails plays a major role in today’s business, one must be keen to preserve the authenticity of mails they sent to the users and fail to do so may result in getting your mail server IP blacklisted and mails ends-up in users junk folder and they die!

This post is not a perfect guide to prevent you from getting blacklisted by the spamcops, but a few tips that could save your “Time” and “Reputation”.

1) All email is filtered against published blacklists of spam servers. Check whether your mailserver IP is already blacklisted : http://www.mxtoolbox.com/. If listed, take necessary actions to remove it. You can temporarily switch the mail server IP to get the mails moving until your regular IP is released.

2) All mails will be filtered based on certain rules. It looks for senders email address, mail subject line and words on the message content. Avoid using blank subject line and using common spammers words like “offer, discount, sale, free etc… There are a lot more..

3) Maintain a regular time intervals while sending mails to a particular sender. This could be setup on exim configuration file. (Needs admin privileges)

4) Enable SPF:
Sender Policy Framework (SPF), is an e-mail validation system designed to prevent e-mail spam by addressing a common vulnerability, source address spoofing. SPF allows administrators to specify which hosts are allowed to send e-mail from a given domain by creating a specific DNS SPF record in the public DNS for that domain. Mail exchangers then use the DNS to check that mail from a given domain is being sent by a host sanctioned by that domain’s administrators.

If SPF record is enabled for a domain, spammers and phishers are less likely to forge e-mails pretending to be from that domain. Spam filters now check for SPF records and hence eliminate the chance of forged mails, spams. Hence an SPF protected domain is less attractive to spammers and phishers and is less likely to be blacklisted by spam filters and ligitimate mails will go through.

SPF keeps the detail of the machine which is only authorized to send mails for that particular domain. This is done by adding additional a TXT record to their existing DNS records. Mail receivers that checks for SPF records check the domain DNS and finds whether the server is allowed to send mails for that domain.

(Read the rest of this entry…)

Tweet This Post

Continue Reading

How to resolve : Internal Server Error when accessing cpanel/WHM/Webmail

Imagine you are accessing your Website WHM or cPanel or Webmail hosted (in your VPS) and what if you see this

How to solve this

Pre-Requisites : You should have Shell access (as root) to the Node server.

Mostly this happens due to inode issue. The inodes allocated to the server maybe full. inode is a data structure on a traditional Unix-style file system such as UFS. An inode stores basic information about a regular file, directory, or other file system object.

Steps

Say we are logged into the Node Server. There we are going to list all the Containers existing on the Hardware Node. By default, only running Containers are shown.

[root@linuxvps1 ~]# vzlist -a
CTID    NPROC STATUS   IP_ADDR        HOSTNAME
100     18  running    100.20.11.111  WEB.EXAMPLE.COM
101     63  running    100.20.12.203  -
102     169 running    100.20.92.182  server2.example.com
103     81  running    100.20.56.169  example.net
78965   236 running    100.20.77.204  server1.example.com

Assume your container is 103. So simply enter into 103 by this command

[root@linuxvps1 ~]# vzctl enter 103

Check if the inodes are full in the container

[root@linuxvps1 ~]# df -i

Most Probably it should display something like this :

Filesystem      Inodes      IUsed        IFree     IUse%      Mounted on
/dev/vzfs       200000      200000       0         100%       /
none            65536       95           65441     1%         /dev

Bang ! The inodes on /dev/vzfs mounted on / are full. You’ve to go nowhere else to see why that error was delivered. A simple solution will work out, you’ve to increase the inode limit. You’ve two options, either through Shell or through the Node Control Panel (like Parallels Infrastructure Manager)

First we will go through Shell mode with this command :

$ vzctl set veid --diskinodes softlimit:hardlimit 

Where ‘veid’ is the VPS ID

root@linuxvps1 ~]# vzctl set 103 --diskinodes 7680100:7680200

Second we are going for Node Control Panel. In the infrastructure, you will see your VPS with a ! sign. That indicates your VPS is having some sort of issue.

So you’ve seen the Alert. Next access the VPS, go to Resources tab. I’m sorry to tell you that I grabbed this screenshot after resolving the issue, but it will help you anyway

You should see the ‘Disk Inodes’ is full under ‘Disk Usage’ category (Here it is not !). Next Click Configure to settle this one

See the Highlighted part, the inodes should be limited to some size. Increase it or remove the check mark to make it unlimited.  Make sure you save the changes. Now try again accessing cpanel/WHM/Webmail. It should load with out any issues.That makes a smile in your face, isn’t it ?

Tweet This Post

Continue Reading

cPanel, postgresql and a default database issue with 8.4.x versions of postgres

The issue was there, because customer wanted a latest version of Postgresql, latest than what cPanel support by default.

Issue reported in the phppGAdmin page was

FATAL:  password authentication failed for user "cPanel_username"
FATAL:  permission denied for database "template1"
DETAIL:  User does not have CONNECT privilege.

(Read the rest of this entry…)

Tweet This Post

Continue Reading