7/21/2015

Ansible freebsd ssh connection hung

Recently I faced with problem which caused almost a week to find right solution

So the problem was that I wasn't able to connect to my freebsd host from ansible server.

so symptoms were like that

sshpass -d6 ssh -C -v -o ControlMaster=auto -o ControlPersist=60s -o ControlPath="/home/michael/.ansible/cp/ansible-ssh-%h-%p-%r" -o StrictHostKeyChecking=no -o GSSAPIAuthentication=no -o PubkeyAuthentication=no -o User=root -o ConnectTimeout=10 192.168.xxx.xxx /bin/csh -c 'LANG=en_US.UTF-8 LC_CTYPE=en_US.UTF-8 /usr/local/bin/python'
[............]

Warning: Permanently added '192.168.2.110' (ECDSA) to the list of known hosts.debug1: SSH2_MSG_NEWKEYS sentdebug1: expecting SSH2_MSG_NEWKEYSdebug1: SSH2_MSG_NEWKEYS receiveddebug1: Roaming not allowed by serverdebug1: SSH2_MSG_SERVICE_REQUEST sentdebug1: SSH2_MSG_SERVICE_ACCEPT receiveddebug1: Authentications that can continue: publickey,keyboard-interactivedebug1: Next authentication method: keyboard-interactive

ansible new-hosts -l freebsd.local -m shell -a 'uptime'  -vvv
<192.168.2.110> ESTABLISH CONNECTION FOR USER: root
<192.168.2.110> REMOTE_MODULE command uptime #USE_SHELL
<192.168.2.110> EXEC sshpass -d6 ssh -C -v -o ControlMaster=auto -o ControlPersist=60s -o ControlPath="/home/michael/.ansible/cp/ansible-ssh-%h-%p-%r" -o StrictHostKeyChecking=no -o GSSAPIAuthentication=no -o PubkeyAuthentication=no -o User=root -o ConnectTimeout=10 192.168.xxx.xxx /bin/sh -c 'LANG=en_US.UTF-8 LC_CTYPE=en_US.UTF-8 /usr/local/bin/python'

and in that state connection could be couple of hours without any output on ansible server or freebsd logs.

The reason of the problem was FreeBSD PAM module 

I had to change line 12 (in my case ) in /etc/pam.d/sshd

was :

auth            required        pam_unix.so             no_warn try_first_pass

become:

auth            required        pam_unix.so             no_warn try_first_pass authtok_prompt=Password:

no sshd restart required

after that changes I was able to get info from my freebsd hosts

 ansible new-hosts -l freebsd.local -m  ping  
freebsd.local | success >> {
    "changed": false, 
    "ping": "pong"
}
ansible new-hosts -l freebsd.local -m shell -a 'uptime'  
freebsd.local | success | rc=0 >>
 2:30PM  up 4 days, 45 mins, 2 users, load averages: 0.11, 0.09, 0.07



1/20/2014

installing ffmpeg-php with php5.3+

the main purpose of this small note is to save your time from the mistakes I've made

so what we have:

centos 5 or 6
php 5.3, 5.4 or 5.5

and great desire to have ffmpeg-php installed on that server :)

so

wget http://sourceforge.net/projects/ffmpeg-php/files/ffmpeg-php/0.6.0/ffmpeg-php-0.6.0.tbz2/download
tar xf ffmpeg-php-0.6.0.tbz2
cd ffmpeg-php-0.6.0

phpize
 ./configure 
 and right before make we will do a magic

1) vi ffmpeg_frame.c
 :%s/PIX_FMT_RGBA32/PIX_FMT_RGB32

2) vi ffmpeg_movie.c
row 311: list_entry *le; TO zend_rsrc_list_entry *le;
row 346: list_entry new_le; TO zend_rsrc_list_entry new_le;
row 360: hashkey_length+1, (void *)&new_le, sizeof(list_entry), TO hashkey_length+1, (void *)&new_le,sizeof(zend_rsrc_list_entry), 


make && make install





5/13/2013

Push to multiple remote repositories from a single local repo in Mercurial

You can set multiple remote repository aliases in the [paths] section of the repository configuration file. This file is in .hg/hgrc, and you would add paths like this:
[paths]
default = http://your-default-repo.com/repo
2nd = http://your-second-repo.com/repo
Then you would run hg push 2nd to push to your second repo and hg push to push to your default repo. 
Unfortunately its not possible to run  hg push * to push to all remote hosts at once, you have to specify 
each push destination one by one.

8/09/2012

view confs without comments

Just to note to myself. how to generate clean configs w/out any comments


sed -e '/^$/d;/^#/d;/^;/d' /etc/php.ini > php.ini.clean

result

root@ ~ # cat /etc/php.ini |wc -l
1249
root@ ~ # cat php.ini.clean |wc -l
182


5/24/2012

{new stable release} Zabbix auto install scripts

!!!NOTE: this scripts only for education/testing/investigation purposes, use on your own risk

I've prepared 2 auto Zabbix 2.0.0  installation scripts, tested on Centos 5.8 and 6.2

https://www.dropbox.com/s/pdekmf5yi6on8o3/zabbix20_el6_install.sh << Centos 6.2
https://www.dropbox.com/s/6sdlerh4ufthys4/zabbix20_el5_install.sh << Centos 5.8

short desc
#!/bin/sh
#
# Centos 6.2 (5.8) zabbix 2 install w/ psql9.1 jabber IPv6 support
#
# known problems:
# zabbix web config not fully configured during installation
# need manually to  setup $ZBX_SERVER  and $ZBX_SERVER_NAME parameters after installation finish
# after installation rpm prepared and it is in /root/rpmbuild/RPMS/
# do not know why but in this version zabbix won't work with DB user other than root had to create role for root
#
# reboot required after installation
#
#
# software distributed under the License is distributed on an "AS IS" BASIS,  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND
# (c) Michael Veselovski support@omgsw.com
# v1: initial release 4/10/2011
# v2: 24/12/2011 new release:
# - psql 9.1 support added
# - db user should be provided during installation
# - zabbix 1.9.8 support added
# v3.1: 15/05/2012 new release
# - script complete rewitten for Centos 6.2 suppport
# - script installing Zabbix 2.0.0rc4 /psql
# v3.2: 15/05/2012 second script for Centos 5.8
# - separate script for Centos 5.8 
# v4: 23/05/2012 script for zabbix 2.0.0
# - same problems as above

5/15/2012

{new release} Zabbix auto install scripts

!!!NOTE: this scripts only for education/testing/investigation purposes, use on your own risk


I've prepared 2 auto Zabbix 2.0.0rc4 installation scripts, tested on Centos 5.8 and 6.2

https://www.dropbox.com/s/unzvemer4itptdt/zabbix2rc4_el5_install.sh
https://www.dropbox.com/s/88nudr29280xm61/zabbix2rc4_el6_install.sh

short desc
#!/bin/sh
#
# Centos 6.2 (5.8) zabbix 2rc4 install w/ psql9.1 jabber IPv6 support
#
# known problems:
# zabbix web config not fully configured during installation
# need manually to  setup $ZBX_SERVER  and $ZBX_SERVER_NAME parameters after installation finish
# after installation rpm prepared and it is in /root/rpmbuild/RPMS/
# do not know why but in this version zabbix won't work with DB user other than root had to create role for root
#
# reboot required after installation
#
#
# software distributed under the License is distributed on an "AS IS" BASIS,  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND
# (c) Michael Veselovski support@omgsw.com
# v1: initial release 4/10/2011
# v2: 24/12/2011 new release:
# - psql 9.1 support added
# - db user should be provided during installation
# - zabbix 1.9.8 support added
# v3.1: 15/05/2012 new release
# - script complete rewitten for Centos 6.2 suppport
# - script installing Zabbix 2.0.0rc4 /psql
# v3.2: 15/05/2012 second script for Centos 5.8
# - separate script for Centos 5.8

5/07/2012

How to upgrade Joomla from 1.5/1.6 to 2.5

BEFORE APPLYING ANY CHANGES YOU SHOULD CREATE A BACKUP

1) Go to http://domain/administrator => Extensions => Install/Uninstall => Upload Package File “com_jupgrade-2.5.1.zip” => Upload File & Install

2) Extensions => Plugin Manager => find “System - Mootools Upgrade” => Enabled “Yes”

3) create “jupgrade”subfolder in RootDirectory (/var/www/public_html/jupgrade), change owner to Apache user and change permissions to “777”

4) Download “http://joomlacode.org/gf/download/frsrelease/16804/73116/Joomla_2.5.3-Stable-Full_Package.zip” (download link: http://joomlacode.org/gf/project/joomla/frs/) to this directory (./jupgrade) and unpack

5) Go to http://domain/administrator => Components => Jupgrade => Parameters:
Distribution => Joomla 2.5
Skip checks => Yes
Skip Download => Yes
Skip Decompress => Yes
Enable migration debug => Yes
Enable Mootools version debug => Yes
DO NOT FORGET “Save” on the top and close this window.

6) Press “START UPGRADE”

7) After upgrade you should check all links. You may do that via “Xenu's Link Sleuth”.
This program works under Fedora, Red Hat, Ubuntu (use Wine) and Windows.
You may download it from “http://home.snafu.de/tilman/XENU.ZIP”, just unzip, install and start under Wine.
1) Go to “File => Check URL”
2) Enter “http://domain/jupgrade/”, NOT http://domain/jupgrade, DO NOT FORGET the last “/”.
(More about this program: http://home.snafu.de/tilman/xenulink.html)

8) If all looks fine - create another subfolder “my_old_joomla” in RootDirectory and move all content FROM RootDirectory to this subdirectory (from /var/www/public_html/ to /var/www/public_html/my_old_joomla) except “jupgrade” and “my_old_joomla” folders.

9) As we can see the RootDirectory is contained only two folders “jupgrade” and “my_old_joomla” you should move all content from /var/www/public_html/jupgrade to /var/www/public_html/

10) That’s all.


(c)  Yan Ternovsky

5/05/2012

Changing tables engine from MyISAM to InnoDB

1. Edit /etc/my.cnf , make sure that:
  
#skip-innodb - commented out
innodb_file_per_table = ON

2. install Percona toolkit

Visit http://www.percona.com/software/percona-toolkit/ to download the latest release of Percona Toolkit. Or, get the latest release from the command line:
 
wget percona.com/get/percona-toolkit.tar.gz

wget percona.com/get/percona-toolkit.rpm

wget percona.com/get/percona-toolkit.deb

You can also get individual tools from the latest release:
 
wget percona.com/get/TOOL 
 
Replace TOOL with the name of any tool.

3. using pt-find convert tables from MyISAM to  InnoDB

pt-find db-name --engine MyISAM --exec "ALTER TABLE %D.%N ENGINE=INNODB" --print

[how to]Windows slow download speed

in case if Windows servers show poor download performance in clouds based on Onapp software its necessary to do the following

Networking: Xen network adapter->Xen network device drivers-> Properties
Configure:  -> Properties->Advanced

In advanced properties should be set:
CheckSumOffload-> Disabled
LargeSendOffload->Disabled

and as usual reboot VPS

1/20/2012

mdadm

1. start raid

mdadm --assemble --scan

2. start raid in case if you do not have /etc/mdadm.conf (should have raid UUID)

mdadm /dev/md0 --assemble -u e6d85d3d:d20ddcfc:c7cxxxxx:6a0xxxx

3. generate /etc/mdadm.conf

 mdadm --detail --scan >> /etc/mdadm.conf

1/10/2012

[mdadm] 2 how-tos

1. how to cancel sync:

in order to cancel raid sync its necessary to left only one drive as active (we are talking about raid1):

 mdadm --fail /dev/md2 /dev/sda3
and  we will receive something like that (in my case it was md2 and /dev/sda3):

 mdadm --detail /dev/md2
/dev/md2:
        Version : 0.90
  Creation Time : Fri Feb  4 21:22:50 2011
     Raid Level : raid1
     Array Size : 1462766336 (1395.00 GiB 1497.87 GB)
  Used Dev Size : 1462766336 (1395.00 GiB 1497.87 GB)
   Raid Devices : 2
  Total Devices : 2
Preferred Minor : 2
    Persistence : Superblock is persistent

    Update Time : Tue Jan 10 18:37:27 2012
          State : clean, degraded
 Active Devices : 1
Working Devices : 1
 Failed Devices : 1
  Spare Devices : 0

           UUID : 71b91f85:bdd01b3f:776c2c25:004bd7b2
         Events : 0.28394

    Number   Major   Minor   RaidDevice State
       0       0        0        0      removed
       1       8       19        1      active sync   /dev/sdb3

       2       8        3        -      faulty spare   /dev/sda3
 /dev/sda3 was set as failed.

2. how to re-add failed disk into existing array and restart scan :)

  • set disk as failed (we already did that in 1st how-to )
  • remove device from array
  • mdadm --remove /dev/md2 /dev/sda3 
  • add device one more time
  • mdadm --add /dev/md2/ /dev/sda3

We can check the status of the arrays on the system with:
watch -n .1 cat /proc/mdstat
 or
mdadm --detail /dev/md0

10/04/2011

Zabbix auto install script

few weeks ago I've noticed that Zabbix team actively developed version 2 of their nice monitoring tool. I decided to install beta version with Postgresql 9.1. Unfortunately Zabbix 1.9.6 does not support Psql 9.1, but with my help :) they fixed that issue. meanwhile I've prepared script which install zabbix from source on clean Centos with Psql 8.4 support.

there is download link to that script https://www.dropbox.com/s/e205kh9vnwyppp8/zabbix_install_centos.sh

this script will be good for testers, sys admins who want to check new features of zabbix :)

9/03/2011

reboot hung server

The "magic SysRq key" provides a way to send commands directly to the kernel through the /proc filesystem. It is enabled via a kernel compile time option, CONFIG_MAGIC_SYSRQ, which seems to be standard on most distributions. First you must activate the magic SysRq option:
    echo 1 > /proc/sys/kernel/sysrq
When you are ready to reboot the machine simply run the following:
    echo b > /proc/sysrq-trigger
This does not attempt to unmount or sync filesystems, so it should only be used when absolutely necessary, but if your drive is already failing then that may not be a concern.

7/14/2011

How to upgrade Postgres 8.4.x to 9.0.x

I've found a couple of manuals where this procedure is described, but I faced with the problem with postgres-libs which were required for my SVN.

so there is my way how to perfom upgrade

1. service postgresql stop
2. rsync -a -v --progress /var/lib/pgsql/ /var/lib/pgsql.old/ #this will backup everything
3. wget http://yum.pgrpms.org/reporpms/9.0/pgdg-centos-9.0-2.noarch.rpm
4. rpm -i pgdg-centos-9.0-2.noarch.rpm
5. /etc/yum.repos.d/CentOS-Base.repo and add 'exclude=postgresql*' to the [base] and [updates] sections
6. yum remove postgresql84*
7. rpm -e postgresql-libs --nodeps
8. yum install postgresql90 postgresql90-server postgresql90-libs postgresql90-contrib postgresql90-devel
9. service postgresql-9.0 initdb
10. export PATH=/usr/pgsql-9.0/bin:$PATH
11. su - postgres ; export PATH=/usr/pgsql-9.0/bin:$PATH ; pg_upgrade --old-datadir "/var/lib/pgsql.old/data" --new-datadir "/var/lib/pgsql/9.0/data" --old-bindir "/usr/bin" --new-bindir "/usr/pgsql-9.0/bin"
12. cp /var/lib/pgsql.old/data/pg_hba.conf /var/lib/pgsql/9.0/data/pg_hba.conf
13. service postgresql-9.0 start
14. vacuumdb --all --analyze-only


Have fun with new Psql version :)

5/31/2011

hackmysql mirror

I've created mirror of hackmysql for personal usage

replace mysql-server with percona server

what is percona-server:

Quote:
Percona Server is an enhanced drop-in replacement for MySQL. With Percona Server,

Your queries will run faster and more consistently.
You will consolidate servers on powerful hardware.
You will delay sharding, or avoid it entirely.
You will save money on hosting fees and power.
You will spend less time tuning and administering.
You will achieve higher uptime.
You will troubleshoot without guesswork.

Does this sound too good to be true? It's not. Percona Server offers breakthrough performance, scalability, features, and instrumentation. Its self-tuning algorithms and support for extremely high-performance hardware make it the clear choice for companies who demand the utmost performance and reliability from their database server


now if you want to have such miracle on your cpanel server you need to do this steps for replacing mysql-server with percona-server:

1. backup VPS.
2. dump all dbs.
3. stop all services on the vps
Code:
/etc/init.d/httpd stop ; /etc/init.d/exim stop ; /etc/init.d/pure-ftpd stop ; /etc/init.d/dovecot stop ; /etc/init.d/cpanel stop ; /etc/init.d/lfd stop ; /etc/init.d/munin-node stop ; /etc/init.d/crond stop ; /etc/init.d/mysql stop ; /etc/init.d/exim stop ;


4. make sure that everything is stopped
5. check what mysql packages are installed

Code:
rpm -qa | grep -i mysql


in my case it was MySQL-client MySQL-devel MySQL-test MySQL-bench MySQL-shared MySQL-server
so I run this command to completely remove mysql from vps
Code:
yum remove MySQL-client MySQL-devel MySQL-test MySQL-bench MySQL-shared MySQL-server


6. add percona repo

Code:
rpm -Uhv http://www.percona.com/downloads/percona-release/percona-release-0.0-1.x86_64.rpm


7. check what packages are available in percona repo

Code:
yum list | grep percona


should be something like

Code:
# yum list | grep percona
percona-release.x86_64                    0.0-1                        installed
Percona-SQL-50-debuginfo.x86_64           5.0.92-b23.85.rhel5          percona
Percona-SQL-client-50.x86_64              5.0.92-b23.85.rhel5          percona
Percona-SQL-devel-50.x86_64               5.0.92-b23.85.rhel5          percona
Percona-SQL-server-50.x86_64              5.0.92-b23.85.rhel5          percona
Percona-SQL-shared-50.x86_64              5.0.92-b23.85.rhel5          percona
Percona-SQL-shared-compat.x86_64          5.1.43-2                     percona
Percona-SQL-test-50.x86_64                5.0.92-b23.85.rhel5          percona
Percona-Server-51-debuginfo.x86_64        5.1.56-rel12.7.224.rhel5     percona
Percona-Server-55-debuginfo.x86_64        5.5.11-rel20.2.115.rhel5     percona
Percona-Server-client-51.x86_64           5.1.56-rel12.7.224.rhel5     percona
Percona-Server-devel-51.x86_64            5.1.56-rel12.7.224.rhel5     percona
Percona-Server-server-51.x86_64           5.1.56-rel12.7.224.rhel5     percona
Percona-Server-shared-51.x86_64           5.1.56-rel12.7.224.rhel5     percona
Percona-Server-test-51.x86_64             5.1.56-rel12.7.224.rhel5     percona
gnuplot.x86_64                            4.2.6-14                     percona
gnuplot-emacs.x86_64                      4.2.6-14                     percona
maatkit.noarch                            7284-1                       percona
xtrabackup.x86_64                         1.6-245.rhel5                percona
xtrabackup-debuginfo.x86_64               1.6-245.rhel5                percona


8. decide what version you need 5.1 or 5.5
in my case I've installed percona 5.5

Code:
# rpm -qa | grep Percona
Percona-Server-client-55-5.5.11-rel20.2.115.rhel5
Percona-Server-server-55-5.5.11-rel20.2.115.rhel5
Percona-Server-shared-55-5.5.11-rel20.2.115.rhel5
Percona-Server-shared-compat-5.5.11-rel20.2.115.rhel5
Percona-Server-devel-55-5.5.11-rel20.2.115.rhel5
Percona-Server-test-55-5.5.11-rel20.2.115.rhel5


9. after Percona-server installation it is necessary to check if it started correctly as some variables in my.cnf are not supported.
10. after successful start it is necessary to run mysql_upgrade.
11. for finishing installation run easyapache and recompile apache/php/etc
12. Go to Server Configuration -> Update Config and set MySQL to "never", save, double check it has been done
13. Test new Apache to be sure server is working

3/25/2011

add json support into Flex4

Adobe itself made a JSON library that you can find at https://github.com/mikechambers/as3corelib Follow these steps to add your JSON library to your Flex project:
  1. Download the .zip archive and extract it. What you need is the .swc file in the lib folder.
  2. In Flex Builder / Flash Builder, go to your project’s properties, and then to the Flex Build Path panel.
  3. Click on Add SWC, and select the as3corelib.swc file
  4. In your project, add the following line: import com.adobe.serialization.json.JSON;
You should be able now to use JSON functionalities in your Flex project :)

12/17/2010

where is my free disk space :)

To list files larger than 10M:
find / -type f -size +10000k -exec ls -lh {} \; | awk '{ print $8 ": " $5 }'

To list files larger than 100M:
find / -type f -size +100000k -exec ls -lh {} \; | awk '{ print $8 ": " $5 }'

11/29/2010

how to convert tables to InnoDB

Using nice tool from Maatkit, we can easy convert tables from any storage engine to other. there is example how to convert from myisam to  innodb

mk-find db-name --engine MyISAM --exec "ALTER TABLE %D.%N ENGINE=INNODB" --print

11/22/2010

how to install php5.3.x & 5.2.x centos

there are 3 steps how to install/upgrade php on centos server/vps

1. install webtatic repo:

rpm -ivh http://repo.webtatic.com/yum/centos/5/`uname -i`/webtatic-release-5-1.noarch.rpm

2. install/upgrade php 5.3.x

yum --enablerepo=webtatic install php
yum --enablerepo=webtatic upgrade php

3. install/upgrade php 5.2.x


yum --enablerepo=webtatic --exclude=php*5.3* update php
yum --enablerepo=webtatic --exclude=php*5.3* install php