Friday, December 30, 2016

File/folder size explorer in Linux



Wanna check out linux disk space usage (and kick out some old files/folders) ?

checkout baobab - yes its very interesting name, which is quite difficult to remember.

of course, its a X based application :)

Thursday, December 29, 2016

resize qemu kvm image size


On HOST MACHINE:
==========================
// adding 2 G space to qemu image
$ qemu-img resize /var/lib/libvirt/images/centos7.0_ceph3.qcow2  +2G
Image resized.


NOW INSIDE VM:
=========================
// Now delete the root / partition and add new partition with updated size.
[root@marvelcentos ~]# fdisk /dev/vda
Welcome to fdisk (util-linux 2.23.2).

Command (m for help): p

Disk /dev/vda: 5368 MB, 5368709120 bytes, 10485760 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk label type: dos
Disk identifier: 0x0009f32f

   Device Boot      Start         End      Blocks   Id  System
/dev/vda1   *        2048     2099199     1048576   83  Linux
/dev/vda2         2099200     6291455     2096128   8e  Linux LVM

Command (m for help): d
Partition number (1,2, default 2): 2
Partition 2 is deleted

Command (m for help): n
Partition type:
   p   primary (1 primary, 0 extended, 3 free)
   e   extended
Select (default p): p
Partition number (2-4, default 2): 2
First sector (2099200-10485759, default 2099200):
Using default value 2099200
Last sector, +sectors or +size{K,M,G} (2099200-10485759, default 10485759):
Using default value 10485759
Partition 2 of type Linux and of size 4 GiB is set

Command (m for help): t
Partition number (1,2, default 2): 2
Hex code (type L to list all codes): 8e
Changed type of partition 'Linux' to 'Linux LVM'

Command (m for help): w
The partition table has been altered!

Calling ioctl() to re-read partition table.

WARNING: Re-reading the partition table failed with error 16: Device or resource busy.
The kernel still uses the old table. The new table will be used at
the next reboot or after you run partprobe(8) or kpartx(8)
Syncing disks.
[root@marvelcentos ~]#
---------------------------
REBOOT VM
# reboot
=========================++
Now in VM MACHINE :
++++++++++++++++++
// extend the physical volume
[root@marvelcentos ~]# pvdisplay
  --- Physical volume ---
  PV Name               /dev/vda2
  VG Name               cl_supercentos
  PV Size               2.00 GiB / not usable 2.00 MiB
  Allocatable           yes (but full)
  PE Size               4.00 MiB
  Total PE              511
  Free PE               0
  Allocated PE          511
  PV UUID               ngQJZP-cOEp-S0DE-7JJX-1MN9-btqJ-zZ8pIH
[root@marvelcentos ~]#
[root@marvelcentos ~]# pvresize /dev/vda2
  Physical volume "/dev/vda2" changed
  1 physical volume(s) resized / 0 physical volume(s) not resized
[root@marvelcentos ~]# pvdisplay
  --- Physical volume ---
  PV Name               /dev/vda2
  VG Name               cl_supercentos
  PV Size               4.00 GiB / not usable 2.00 MiB
  Allocatable           yes
  PE Size               4.00 MiB
  Total PE              1023
  Free PE               512
  Allocated PE          511
  PV UUID               ngQJZP-cOEp-S0DE-7JJX-1MN9-btqJ-zZ8pIH
[root@marvelcentos ~]#
[root@marvelcentos ~]#
-------------------------------------
// Extend the logical volume with *updated* physical volume
[root@marvelcentos ~]# lvextend  /dev/mapper/cl_supercentos-root  /dev/vda2 
  Size of logical volume cl_supercentos/root changed from 1.70 GiB (434 extents) to 3.70 GiB (946 extents).
  Logical volume cl_supercentos/root successfully resized.
[root@marvelcentos ~]#
[root@marvelcentos ~]# df -h
Filesystem                       Size  Used Avail Use% Mounted on
/dev/mapper/cl_supercentos-root  1.7G  1.4G  318M  82% /
devtmpfs                         910M     0  910M   0% /dev
tmpfs                            920M     0  920M   0% /dev/shm
tmpfs                            920M   17M  904M   2% /run
tmpfs                            920M     0  920M   0% /sys/fs/cgroup
/dev/vda1                       1014M  184M  831M  19% /boot
tmpfs                            184M     0  184M   0% /run/user/0
[root@marvelcentos ~]#
[root@marvelcentos ~]#

// Extend the underlying partition too
[root@marvelcentos ~]# xfs_growfs /dev/mapper/cl_supercentos-root  
meta-data=/dev/mapper/cl_supercentos-root isize=512    agcount=4, agsize=111104 blks
         =                       sectsz=512   attr=2, projid32bit=1
         =                       crc=1        finobt=0 spinodes=0
data     =                       bsize=4096   blocks=444416, imaxpct=25
         =                       sunit=0      swidth=0 blks
naming   =version 2              bsize=4096   ascii-ci=0 ftype=1
log      =internal               bsize=4096   blocks=2560, version=2
         =                       sectsz=512   sunit=0 blks, lazy-count=1
realtime =none                   extsz=4096   blocks=0, rtextents=0
data blocks changed from 444416 to 968704

// See the results :)
[root@marvelcentos ~]# df -h
Filesystem                       Size  Used Avail Use% Mounted on
/dev/mapper/cl_supercentos-root  3.7G  1.4G  2.4G  38% /
devtmpfs                         910M     0  910M   0% /dev
tmpfs                            920M     0  920M   0% /dev/shm
tmpfs                            920M   17M  904M   2% /run
tmpfs                            920M     0  920M   0% /sys/fs/cgroup
/dev/vda1                       1014M  184M  831M  19% /boot
tmpfs                            184M     0  184M   0% /run/user/0
[root@marvelcentos ~]#

====================================

Note: On Centos, file system is mounted by xfs file system , you cannot use resize2fs,
rather you should use xfs_growfs.

reboot after partition update is quite important ( you may call partprobe too instead of reboot).


Wednesday, December 28, 2016

Android - open notification panel on the lock screen




 Opening the notification panel on the lock screen:


Enable this feature by going to Settings > Notifications > Advanced settings and slide the Enable on lock screen switch to the right.

source: http://webcazine.com/12040/how-to-display-notifications-on-xiaomi-phones-lock-screen-2/


ssh bad owner or permissions on config


While doing ssh to a hostname (with a custom configuration file named ~/.ssh/config) faced this error:
ssh bad owner or permissions on config

Solution:
chown $USER ~/.ssh/config
chmod 644 ~/.ssh/config
Because of the potential for abuse, this file must have strict permissions: read/write for the user, and not writable by others. It may be group-writable provided that the group in question contains only the user.



source: https://serverfault.com/questions/253313/ssh-hostname-returns-bad-owner-or-permissions-on-ssh-config/710453#710453

cloning a qemu VM and removing old details present in clone



Cloning the vm:

First clone the VM using virt-manager UI. (launch # sudo virt-manager, the click on the VM to clone, and right click and select clone).

Assign a new name while cloning.

Ensure the new VM is shutdown and then run virt-sysprep as follows

Then, use virt-sysprep as follows:

virt-sysprep -d <new name of clone> --hostname <new hostname> --root-password password:<new root password>

Here,
-d => decontextualize

Look at man virt-sysprep for more options available.

-------------------
How do you get new name of the clone?

It is the name field available in the xml file.
-------------------

Example:
# virt-sysprep -d node2  --hostname node2 --root-password password:testpassword  
[   0.0] Examining the guest ...
[   2.8] Performing "abrt-data" ...
[   2.8] Performing "bash-history" ...
[   2.8] Performing "blkid-tab" ...
[   2.8] Performing "crash-data" ...
[   2.8] Performing "cron-spool" ...
[   2.8] Performing "dhcp-client-state" ...
[   2.8] Performing "dhcp-server-state" ...
[   2.8] Performing "dovecot-data" ...
[   2.8] Performing "logfiles" ...
[   2.9] Performing "machine-id" ...
[   2.9] Performing "mail-spool" ...
[   2.9] Performing "net-hostname" ...
[   2.9] Performing "net-hwaddr" ...
[   2.9] Performing "pacct-log" ...
[   2.9] Performing "package-manager-cache" ...
[   2.9] Performing "pam-data" ...
[   2.9] Performing "puppet-data-log" ...
[   2.9] Performing "rh-subscription-manager" ...
[   2.9] Performing "rhn-systemid" ...
[   2.9] Performing "rpm-db" ...
[   2.9] Performing "samba-db-log" ...
[   2.9] Performing "script" ...
[   2.9] Performing "smolt-uuid" ...
[   2.9] Performing "ssh-hostkeys" ...
[   2.9] Performing "ssh-userdir" ...
[   2.9] Performing "sssd-db-log" ...
[   2.9] Performing "tmp-files" ...
[   2.9] Performing "udev-persistent-net" ...
[   2.9] Performing "utmp" ...
[   2.9] Performing "yum-uuid" ...
[   2.9] Performing "customize" ...
[   2.9] Setting a random seed
[   2.9] Setting the hostname: node2
[   2.9] Setting passwords
[   4.9] Performing "lvm-uuids" ...

=======================================



Friday, December 16, 2016

disable automount/running programs on media insertion - fedora 22



To disable any automounting / running program when android mobile(or any media) is inserted:

Window -> type "Details" -> "removable media"  -> check "Never prompt or start programs on media insertion"


Wednesday, December 7, 2016

sourcing vs executing a script

source a script vs executing(or running) a script, what is the difference?

short:
sourcing will run the commands in the current shell process. executing will run the commands in a new shell process.

long:
sourcing :
Sourcing the script does not create a new shell. All commands are run in the current shell and changes to the environment take effect in the current shell.

It's as if you typed the contents of the script.

 running:
The changes were made in a new shell. The current shell spawned a new shell to run the script. The script is running in the new shell and all changes to the environment take effect in the new shell. After the script is done the new shell is destroyed. All changes to the environment in the new shell are destroyed with the new shell. Only the output text is printed in the current shell.




link :
http://superuser.com/questions/176783/what-is-the-difference-between-executing-a-bash-script-and-sourcing-a-bash-scrip

 

Sunday, December 4, 2016

failed to find romfile "efi-virtio.rom" - qemu failed to start - solution


Started facing this error:
# virsh create centos7.0.xml
error: Failed to create domain from centos7.0.xml
error: internal error: process exited while connecting to monitor: 2016-12-04T16:20:01.524386Z qemu-system-x86_64: -device virtio-net-pci,netdev=hostnet0,id=net0,mac=52:54:00:2e:7b:93,bus=pci.0,addr=0x3: failed to find romfile "efi-virtio.rom"
2016-12-04T16:20:01.524477Z qemu-system-x86_64: -device virtio-net-pci,netdev=hostnet0,id=net0,mac=52:54:00:2e:7b:93,bus=pci.0,addr=0x3: Device 'virtio-net-pci' could not be initialized

Then fixed all the links of rom files as below:

root@gant qemu$ ls -l efi*.rom
lrwxrwxrwx 1 root root 24 Jun 22 20:08 efi-e1000.rom -> ../ipxe.efi/8086100e.rom
lrwxrwxrwx 1 root root 24 Jun 22 20:08 efi-ne2k_pci.rom -> ../ipxe.efi/10ec8029.rom
lrwxrwxrwx 1 root root 24 Jun 22 20:08 efi-pcnet.rom -> ../ipxe.efi/10222000.rom
lrwxrwxrwx 1 root root 24 Jun 22 20:08 efi-rtl8139.rom -> ../ipxe.efi/10ec8139.rom
lrwxrwxrwx 1 root root 24 Jun 22 20:08 efi-virtio.rom -> ../ipxe.efi/1af41000.rom
root@gant qemu$
root@gant qemu$
root@gant qemu$ ln -sf   ../ipxe/8086100e.rom efi-e1000.rom
root@gant qemu$ ln -sf   ../ipxe/10ec8029.rom efi-ne2k_pci.rom
root@gant qemu$ ln -sf ../ipxe/10222000.rom efi-pcnet.rom
root@gant qemu$ ln -sf  ../ipxe/10ec8139.rom efi-rtl8139.rom
root@gant qemu$ ln -sf  ../ipxe/1af41000.rom efi-virtio.rom
root@gant qemu$
root@gant qemu$ ls -l efi*.rom
lrwxrwxrwx 1 root root 20 Dec  4 22:32 efi-e1000.rom -> ../ipxe/8086100e.rom
lrwxrwxrwx 1 root root 20 Dec  4 22:32 efi-ne2k_pci.rom -> ../ipxe/10ec8029.rom
lrwxrwxrwx 1 root root 20 Dec  4 22:32 efi-pcnet.rom -> ../ipxe/10222000.rom
lrwxrwxrwx 1 root root 20 Dec  4 22:32 efi-rtl8139.rom -> ../ipxe/10ec8139.rom
lrwxrwxrwx 1 root root 20 Dec  4 22:32 efi-virtio.rom -> ../ipxe/1af41000.rom
root@gant qemu$


Saturday, December 3, 2016

பாரதி பாடல் - சின்னஞ் சிறு கிளியே

சின்னஞ் சிறு கிளியே - கண்ணம்மா! 
செல்வக் களஞ்சியமே! 
என்னைக் கலி தீர்த்தே - உலகில் 
ஏற்றம் புரிய வந்தாய்! 
பிள்ளைக் கனியமுதே - கண்ணம்மா 
பேசும்பொற் சித்திரமே! 
அள்ளி யணைத்திடவே - என் முன்னே 
ஆடி வருந்தேரே ! 

ஓடி வருகையில் - கண்ணம்மா 
உள்ளங் குளிரு தடீ! 
ஆடித்திரிதல் கண்டால் - உன்னைப்போய் 
ஆவி தழுவு தடீ! 
உச்சி தனை முகந்தால் - கருவம் 
ஓங்கி வளரு தடீ! 
மெச்சி யுனை யூரார் - புகழ்ந்தால் 
மேனி சிலிர்க்கு தடீ! 

கன்னத்தில் முத்தமிட்டால் - உள்ளந்தான் 
கள்வெறி கொள்ளு தடீ! உன்னைத் தழுவிடிலோ - கண்ணம்மா 
உன்மத்த மாகு தடீ! 
சற்றுன் முகஞ் சிவந்தால் - மனது 
சஞ்சல மாகு தடீ! 
நெற்றி சுருங்கக் கண்டால் - எனக்கு 
நெஞ்சம் பதைக்கு தடீ! 

உன்கண்ணில் நீர்வழிந்தால் - என்நெஞ்சில் 
உதிரங் கொட்டு தடீ! 
என் கண்ணிற் பாவையன்றோ? - கண்ணம்மா! 
என்னுயிர் நின்ன தன்றோ? 
சொல்லு மழலையிலே - கண்ணம்மா! 
துன்பங்கள் தீர்த்திடு வாய்; 
முல்லைச் சிரிப்பாலே - எனது 
மூர்க்கந் தவிர்த்திடு வாய். 

இன்பக் கதைக ளெல்லாம் - உன்னைப்போல் 
ஏடுகள் சொல்வ துண்டோ? 
அன்பு தருவதிலே - உனைநேர் 
ஆகுமோர் தெய்வ முண்டோ? 
மார்பில் அணிவதற்கே -உன்னைப்போல் 
வைர மணிக ளுண்டோ? 
சீர்பெற்று வாழ்வதற்கே -உன்னைப் போல் 
செல்வம் பிறிது முண்டோ?


// என்னைக் கலி தீர்த்தே - எனது துன்பங்களையெல்லாம் ஒழித்து


Note:

Updated.
இருந்தபோதிலும், இதில் எழுத்து பிழை இருக்க வாய்ப்பு உள்ளது.
பாரதியார் புத்தகத்தில் பிழையற்ற பாடல் இருக்கும்.

இதை  பார்வைக்கு  மட்டுமே பயன்படுத்தவும்.
நன்றி!


Monday, November 21, 2016

group uninstall of packages - centos



So you know how to install a group of packages
 yum groupinstall "X Window System"

To uninstall the same, use :
 yum groupremove "X Window System"


Thursday, November 10, 2016

duplicate lines in a file - command line


Lets say you have a file with some duplicate lines.
How do you figure out them using Linux command line utilities?


 For example, file contents as this:
~$ cat test.txt
line 1
line 2
line 3
line 4
line 5
line 5
line 6
line 7


Now use the command to get the count of duplicate lines.
so, "line 5" is repeated twice.

~$ sort test.txt  | uniq -c
      1 line 1
      1 line 2
      1 line 3
      1 line 4
      2 line 5
      1 line 6
      1 line 7

To get that specific line alone use -cd
~$ sort test.txt  | uniq -cd
      2 line 5

Tuesday, November 8, 2016

markdown editor - with preview


There may be many markdown editors available.

but atom from github seems to be the winner :)

first open the file using File-> open.

then, go to packages -> markdown preview -> toggle preview.

Awesome..isn't it?

Monday, November 7, 2016

tldr - simplified man pages

man pages for linux commands are quite lengthy in nature.
They try to cover all aspects of a command.

How about a some simple examples for a linux command.

Welcome to tldr - simplified man page for linux command

Install as below:

sudo yum install npm
npm install -g tldr
tldr --update

Usage is like:

# tldr ls

  ls

  List directory contents.

  - List files one per line:
    ls -1

  - List all files, including hidden files:
    ls -a

  - Long format list (permissions, ownership, size and modification date) of all files:
    ls -la

  - Long format list with size displayed using human readable units (KB, MB, GB):
    ls -lh

  - Long format list sorted by size (descending):
    ls -lS

  - Long format list of all files, sorted by modification date (oldest first):
    ls -ltr




Monday, October 17, 2016

copy list of files to a directory



copy a list of files mentioned in a text file into a directory.

cat /tmp/test.txt  | xargs -I %  cp % /tmp/to_copy/


Here:
test.txt contains list of files to be copied.
/tmp/to_copy/ is the destination folder

Tuesday, September 27, 2016

yahoo mail retreival from gmail

I have configured yahoo mail retrieval into gmail account via POP settings.
But started facing this error in gmail:
Getting error, Server denied POP3 access for the given username and password. POP access is limited to Yahoo Mail Plus subscribers.?



Although I tried configuring it again, yahoo login fails inside gmail.



Figured out, you need to enable "Allow apps that use less secure sign-in" 
in yahoo security settings.

Then you can configure yahoo account from inside gmail. J :)


Monday, September 26, 2016

Control Android mobile from PC


Just stumbled across this app: airdroid


You can control just about anything in your Mobile from PC.

Install app from play store.

Then, visit this url in your PC:
http://web.airdroid.com
and
then, scan the QR code using "airdroid" app(from mobile)..thats it.


Now you can control your mobile from www.web.airdroid.com(from your pc).

Possible to:
Download/Upload of files, Controlling camera, uninstall of app, images view, etc.,


Thursday, September 22, 2016

convert pdf to image format or viceversa






convert pdf to png(image) file:




pdftoppm -rx 300 -ry 300 -png file.pdf prefix



source:
http://askubuntu.com/questions/50170/how-to-convert-pdf-to-image/50180


===============================

To convert image(s) to pdf use:

convert Image1.jpg Image2.jpg  myfile.pdf

So, myfile.pdf will contain both the image files in pdf format.

================================

Also, to reduce the size of the image by 50%

convert  -resize 50% source.jpg  destination.jpg

Note: it can be png format too.

===================================

Wednesday, September 7, 2016

cockpit project - view system info in browser



Cockpit software is to view your system info. in a browser.
Information like CPU, Memory, Networking, Storage, Users, etc.,


Install and setup instructions for Fedora :

1. Install like:
dnf install cockpit.x86_64

2. Then, Enable it like:
sudo systemctl enable  cockpit.socket

3. Start cockpit service
systemctl start  cockpit.socket

4. Enable firewall access
firewall-cmd --add-service=cockpit

5. Make it permanent
firewall-cmd --add-service=cockpit --permanent

6. Now open the browser and access it GUI:
https://ip-address-of-machine:9090


 You can access following components:

Friday, August 19, 2016

read of 0 byte file



What will happen if you read a file with 0 bytes?

say cat /dev/null > file

do a read on the file - what it should return -
So you expect error ??

nope!!


It returns zero, which is success for read but no buffer is stored.



I had some tough time debugging this one.

So, Aware & Beware of this :)

color chooser website



color chooser :)

Getting hex codes for colors:

http://www.huecode.com/


samba - verify correctness of samba conf file



// check an smb.conf configuration file for internal correctness

#testparm




google calendar get remainder on same day


Usually while setting a google calendar notification, I wish to get a reminder
on the same day(birthday for example).

But, it pops up a day before :)

Then, figured out that using "0" day before while setting notification will help my case :))

Sharing a command output with others


wanna share a command output to others?

Answer: use netcat

 How?
First install netcat. package name is
nmap-ncat.x86_64

Then, you can get the output of a command to a browser liike below:

===========================
$ service smb status | nc termbin.com 9999
Redirecting to /bin/systemctl status  smb.service
http://termbin.com/4bpd
===========================

So, open the link in browser:
http://termbin.com/4bpd

Its contents are as below:
===================
● smb.service - Samba SMB Daemon
   Loaded: loaded (/usr/lib/systemd/system/smb.service; enabled; vendor preset: disabled)
   Active: active (running) since Wed 2016-08-17 21:04:42 IST; 1 day 17h ago
 Main PID: 2055 (smbd)
   Status: "smbd: ready to serve connections..."
   CGroup: /system.slice/smb.service
           ├─2055 /usr/sbin/smbd
           └─2057 /usr/sbin/smbd
Warning: Journal has been rotated since unit was started. Log output is incomplete or unavailable.
=================================

You can share this link with others.

Sunday, August 14, 2016

debugging using gstack


       gstack - print a stack trace of a running process

       gstack <pid of the process>

       gstack  attaches  to the active process named by the pid on the command
       line, and prints out an execution stack trace.

       If the process is part of a thread group, then gstack will print out  a
       stack trace for each of the threads in the group.

       Ensure it is not stripped of symbols to get trace.

Wednesday, July 20, 2016

failed with error -2 iwlwifi drivers


Installed a new kernel.

Started facing issues with intel wireless drivers iwlwifi with new kernel booted.

Errors are as below:
===============================
[ 2149.026974] Intel(R) Wireless WiFi driver for Linux
[ 2149.026976] Copyright(c) 2003- 2015 Intel Corporation
[ 2149.027291] iwlwifi 0000:03:00.0: Direct firmware load for iwlwifi-7260-17.ucode failed with error -2
[ 2149.027298] iwlwifi 0000:03:00.0: Direct firmware load for iwlwifi-7260-16.ucode failed with error -2
[ 2149.027303] iwlwifi 0000:03:00.0: Direct firmware load for iwlwifi-7260-15.ucode failed with error -2
[ 2149.027308] iwlwifi 0000:03:00.0: Direct firmware load for iwlwifi-7260-14.ucode failed with error -2
[ 2149.027313] iwlwifi 0000:03:00.0: Direct firmware load for iwlwifi-7260-13.ucode failed with error -2
[ 2149.027314] iwlwifi 0000:03:00.0: request for firmware file 'iwlwifi-7260-13.ucode' failed.
[ 2149.027316] iwlwifi 0000:03:00.0: no suitable firmware found!
===============================


After some googling (some people use the word RnD ;), what an abuse ) ,figured out that one need to install firmware package separately.

So,
# dnf install    iwl7260-firmware.noarch -y

and it fixed the issue :)


Wednesday, June 22, 2016

top output on all cpu cores

wanna check 'top' command on all cpu cores?

first type
#top

then press '1' over top output

=================================

Also, wanna check which specific CPU core a specific process is running?

check this out:

# ps o pid,psr,comm `pidof gvim`
  PID PSR COMMAND
12063   7 gvim


so, 'gvim' command is running on cpu core '7'.
----------------------------------------------------------------------------------

want to check every process and the core in which it is running?

# ps -eo psr,comm,pid




Monday, May 23, 2016

yum undo installation - howto

Get the package install/uninstall history like:
yum history list

Now, undo specific history:
yum history undo <number>

Where <number>, is taken from ID number in "yum history list".



Thursday, May 5, 2016

python-cheetah in rhel7

Facing error related to python-cheetah ( actually a yum installation error during cinder installation).

You need to enable as below:

Source Link:
https://www.rdoproject.org/documentation/repositories/

Enabling the Optional, Extras, and RH Common channels on RHEL

If using RHEL, then RDO needs the OptionalExtras, and RH Common channels to be enabled:

$ sudo subscription-manager repos --enable rhel-7-server-optional-rpms

$ sudo subscription-manager repos --enable rhel-7-server-extras-rpms

$ sudo subscription-manager repos --enable=rhel-7-server-rh-common-rpms


Now, try installing python-cinder.noarch , it should work.
========================================

gluster geo-replication with non-root user


A quick guide to setup non-root geo-replication setup:


mount broker setup :
  
Run the below commands in SLAVE Host

// Assumption:  geoacct user added (with geogrp as group).

// create a directory as below:
mkdir /var/mountbroker_root

// change permission as below - ( ensure /var directory is having default permission )
chmod 0711 /var/mountbroker_root


gluster system:: execute mountbroker opt mountbroker-root  /var/mountbroker_root

// geogrp is group of geoacct user.
gluster system:: execute mountbroker opt geo-replication-log-group geogrp

gluster system:: execute mountbroker opt rpc-auth-allow-insecure on

// tv2 is slave volume , geoacct is slave user to which session is created
gluster system:: execute mountbroker user geoacct tv2

// Now restart glusterd - to make effect of above settings
killall glusterd; glusterd -LDEBUG

// check glusterd.vol file  - (optional)  just to ensure above settings are in place

Run at MASTER Host: (as root user) 
ssh-copy-id  geoacct@gfvm4
where gfvm4 is Slave Host

Now Run at Slave Host:
/usr/local/libexec/glusterfs/set_geo_rep_pem_keys.sh geoacct tv1 tv2

// To review the settings at Slave Host: (optional)
gluster system:: execute mountbroker info

Now setup geo-replication at Master Host:
gluster volume geo-replication tv1 geoacct@gfvm4::tv2 create push-pem

// Check status - It should be created:
gluster volume geo-replication tv1 geoacct@gfvm4::tv2 status

// Start the session:
gluster volume geo-replication tv1 geoacct@gfvm4::tv2 start

// Check status  - it should be Started :
gluster volume geo-replication tv1 geoacct@gfvm4::tv2 status

// If required , you can stop session as below:
gluster volume geo-replication tv1 geoacct@gfvm3::tv2 stop

// You can delete session as below , if you wish so :) :
gluster volume geo-replication tv1 geoacct@gfvm3::tv2 delete


Tuesday, April 19, 2016

Sharding volume in gluster - simple example



Sharding volume:
=============





$ glusterd -LDEBUG

$ gluster volume create tv1 gant:/opt/volume_test/tv_1/b1 gant:/opt/volume_test/tv_1/b2 force
volume create: tv1: success: please start the volume to access data

$ gluster volume start tv1
volume start: tv1: success

$ mount -t glusterfs gant:/tv1 /mnt/master/


$ gluster volume get tv1 features.shard
Option                                  Value                                  
------                                  -----                                  
features.shard                          off                                    

$ gluster volume get tv1 features.shard-block-size
Option                                  Value                                  
------                                  -----                                  
features.shard-block-size               4MB                                    


$ gluster volume set tv1 features.shard-block-size 10MB
volume set: success


$ gluster volume get tv1 features.shard-block-size
Option                                  Value                                  
------                                  -----                                  
features.shard-block-size               10MB                                   

$ gluster volume set tv1 features.shard on
volume set: success


$ mount | grep glusterfs
gant:/tv1 on /mnt/master type fuse.glusterfs (rw,relatime,user_id=0,group_id=0,default_permissions,allow_other,max_read=131072)


$ dd if=/dev/urandom of=testfile count=12 bs=6MB
12+0 records in
12+0 records out
72000000 bytes (72 MB) copied, 4.96613 s, 14.5 MB/s

$ /root/brick_content.sh tv1
Bricks Present:
/opt/volume_test/tv_1/b1
/opt/volume_test/tv_1/b2

Contents of Bricks:
/opt/volume_test/tv_1/b1:
total 0

/opt/volume_test/tv_1/b2:
total 10244
-rw-r--r-- 2 root root 10485760 Apr 19 16:09 testfile

$ ls -ls
total 70313
70313 -rw-r--r-- 1 root root 72000000 Apr 19 16:10 testfile

$ ls -ls /opt/volume_test/tv_1/*/
/opt/volume_test/tv_1/b1/:
total 0

/opt/volume_test/tv_1/b2/:
total 10244
10244 -rw-r--r-- 2 root root 10485760 Apr 19 16:09 testfile

$ ls -ls /opt/volume_test/tv_1/*/.shard/ 
/opt/volume_test/tv_1/b1/.shard/:
total 60076
10240 -rw-r--r-- 2 root root 10485760 Apr 19 16:09 e93bb270-2d70-40da-b74b-1ac4635fb0e3.1
10240 -rw-r--r-- 2 root root 10485760 Apr 19 16:10 e93bb270-2d70-40da-b74b-1ac4635fb0e3.2
10240 -rw-r--r-- 2 root root 10485760 Apr 19 16:10 e93bb270-2d70-40da-b74b-1ac4635fb0e3.3
10240 -rw-r--r-- 2 root root 10485760 Apr 19 16:10 e93bb270-2d70-40da-b74b-1ac4635fb0e3.4
10240 -rw-r--r-- 2 root root 10485760 Apr 19 16:10 e93bb270-2d70-40da-b74b-1ac4635fb0e3.5
 8876 -rw-r--r-- 2 root root  9085440 Apr 19 16:10 e93bb270-2d70-40da-b74b-1ac4635fb0e3.6

/opt/volume_test/tv_1/b2/.shard/:
total 0
$
$ bc
bc 1.06.95
Copyright 1991-1994, 1997, 1998, 2000, 2004, 2006 Free Software Foundation, Inc.
This is free software with ABSOLUTELY NO WARRANTY.
For details type `warranty'.
10485760*5
52428800
52428800+9085440+10485760
72000000
(interrupt) Exiting bc.
$
$
$ gluster volume info tv1

Volume Name: tv1
Type: Distribute
Volume ID: d66102b6-13c1-491a-912e-206d809573de
Status: Started
Number of Bricks: 2
Transport-type: tcp
Bricks:
Brick1: gant:/opt/volume_test/tv_1/b1
Brick2: gant:/opt/volume_test/tv_1/b2
Options Reconfigured:
features.shard: on
features.shard-block-size: 10MB
transport.address-family: inet
performance.readdir-ahead: on
$
$

Sunday, April 17, 2016

cdrom issues in fedora



After a looong time, tried to play some songs from cdrom  ;)

faced these errors:

[146512.870430] sr 5:0:0:0: [sr0] tag#12 FAILED Result: hostbyte=DID_OK driverbyte=DRIVER_SENSE
[146512.870443] sr 5:0:0:0: [sr0] tag#12 Sense Key : Illegal Request [current]
[146512.870445] sr 5:0:0:0: [sr0] tag#12 Add. Sense: Illegal mode for this track
[146512.870447] sr 5:0:0:0: [sr0] tag#12 CDB: Read(10) 28 00 00 05 68 1e 00 00 02 00
[146512.870448] blk_update_request: I/O error, dev sr0, sector 1417336
[146513.233773] sr 5:0:0:0: [sr0] tag#14 FAILED Result: hostbyte=DID_OK driverbyte=DRIVER_SENSE
[146513.233776] sr 5:0:0:0: [sr0] tag#14 Sense Key : Illegal Request [current]
[146513.233779] sr 5:0:0:0: [sr0] tag#14 Add. Sense: Illegal mode for this track
[146513.233780] sr 5:0:0:0: [sr0] tag#14 CDB: Read(10) 28 00 00 05 68 1f 00 00 01 00
[146513.233782] blk_update_request: I/O error, dev sr0, sector 1417340
[146513.233785] Buffer I/O error on dev sr0, logical block 354335, async page read


confused, googled, and what not?


Then tried mounting manually, voila it worked :)

mount /dev/sr0 /mnt/cdrom -o ro




gnome extensions in fedora

checkout these gnome extension and switch ON/OFF according to your needs.

This internally changes your system state :) ..awesome..isn't it?

https://extensions.gnome.org/local/


My favourites:

AlternateTab

Removable Drive Menu





Wednesday, March 16, 2016

check any update on a webpage


So, you are interested in a webpage and check whether it is updated?

How do you check this?
you bookmark it and visit as and when required to see whether there is any update.


Now, there is a nice website : visualping.io
Just enter the webpage you wish to watch (for any changes happened) and set an alert with your mail id.

visualping  notifies you with an email when there is a change.



Tuesday, March 1, 2016

customize bash prompt color


change the color of the bash terminal string: 

export PS1="\e[0;32m[\u@\h \W]\$ \e[m"

[updated]
export PS1="\[\e[0;32m\]\u@\h:\W\$ \[\e[0m\]"

set the above in ~/.bashrc file.

0;32 -> green color


more here:
http://www.cyberciti.biz/faq/bash-shell-change-the-color-of-my-shell-prompt-under-linux-or-unix/


Friday, February 26, 2016

Change resolution and other display settings - Fedora



arandr - GUI app to setup resolution, quite useful for multi monitor settings.

install like # yum install arandr.noarch -y


Tuesday, February 16, 2016

git show commit

// show the first two patches in git branch
git show HEAD -2


// show the top one
git show HEAD
equivalent to
git show HEAD -1

Friday, February 12, 2016

interrupt sleep in a bash script


For testing you have added sleep command in a script.

Now, you want to interrupt the sleep and continue to next statement.

How do you do it?

 Method 1:
If sleep is only done in your script alone, you can do this:
pkill sleep
 
Method 2: (need changes in script) 

--------------------- // this is your script
# write the current session's PID to file
echo $$ >> myscript.pid

# go to sleep for a long time
sleep 1000

---------------------
# pkill -P $(<myscript.pid) sleep 


Ref:
http://askubuntu.com/questions/575704/how-can-i-wake-a-sleeping-bash-script

Sunday, February 7, 2016

syntastic vim plugin



Quite useful that it checks syntax while editing the file.

I can see it in action for both Python and C.


Refer below:
https://github.com/scrooloose/syntastic

and
refer step 2 and 3 in README for installation and recommended settings.


Thursday, February 4, 2016

filesystem type of a file



So, you have a file and want to know filesystem it is mounted on
do this:

df -T /path/to/the/file 

# df -T ~/test.c
Filesystem                      Type 1K-blocks    Used Available Use% Mounted on
/dev/mapper/fedora--server-root ext4  27187420 3801608  21981704  15% /



Monday, February 1, 2016

renaming (existing) hardlinks



Lets say,  We have two files testfile123 and testfile456
#touch testfile123
#touch testfile456

#unalias mv
#mv testfile123 testfile456

strace shows this:
access("testfile456", W_OK)             = 0
rename("testfile123", "testfile456")    = 0
------------------------------
Lets say, we have two hard links

#touch testfile
#ln testfile testfile_hlink1
#ln testfile testfile_hlink2

#mv testfile_hlink1 testfile_hlink2

access("test_hlink1", W_OK)             = 0
unlink("test_hlink2")                   = 0
---------------------------------
So, in case of hardlink, rename system call itself is not called at all (as it is the same file and moving contents from one hardlink to another does not make sense).

man 2 rename :
       If  oldpath  and  newpath are existing hard links referring to the same
       file, then rename() does nothing, and returns a success status.

Thanks to Aravinda  for pointing me this.

Sunday, January 24, 2016

64 bit maximum address space



32 bit processeor have 4GB address space. 2 pow 32 = 4 GB (4294967296)

Is that mean 64 bit can access can access 2 pow 64 = 16 exabytes ??

not really

1. 64 bit does not mean all the bits of those points are actually used.
x86_64 -> 48 bit address lines. -> theoreitically 256 terabytes of RAM.

2. mother board have their own limits how much RAM can be supported.

Physically we can only have few slots ;)

3. OS may have internal limitation on how much it can support.

Linux allows 128 TB of virtual address space per process on x86_64.

Source: http://superuser.com/questions/168114/how-much-memory-can-a-64bit-machine-address-at-a-time
======================================================


installing glusterfs debuginfo package in fedora

Go to repo directory:
cd /etc/yum.repos.d/

wget this file:
wget http://download.gluster.org/pub/gluster/glusterfs/LATEST/Fedora/glusterfs-fedora.repo

Now, search for debuginfo:
yum search glusterfs-debuginfo

Install the package:

yum install glusterfs-debuginfo.x86_64 -y


You are done :)





Tuesday, January 12, 2016

certificate signing using TLS




# Generate private key 
openssl genrsa -out ca.key 2048 

# Generate CSR 
openssl req -new -key ca.key -out ca.csr

# Generate Self Signed Key
openssl x509 -req -days 365 -in ca.csr -signkey ca.key -out ca.crt

# Copy the files to the correct locations
cp ca.crt /etc/pki/tls/certs
cp ca.key /etc/pki/tls/private/ca.key
cp ca.csr /etc/pki/tls/private/ca.csr
 
 
/etc/httpd/conf.d/ssl.conf 
SSLCertificateFile /etc/pki/tls/certs/ca.crt 
SSLCertificateKeyFile /etc/pki/tls/private/ca.key
 
httpd restart
 
 
open in browser like : https:// 
 
This is working through 443 port number (Refer  /etc/httpd/conf.d/ssl.conf  )

There are NO changes required in ( /etc/httpd/conf/httpd.conf )
 
Ref:
https://wiki.centos.org/HowTos/Https

Monday, January 11, 2016

volume files in gluster

===================================
# ls /var/lib/glusterd/vols/tv1/*.vol -l
/var/lib/glusterd/vols/tv1/trusted-tv1.tcp-fuse.vol     //client volume file (1)

/var/lib/glusterd/vols/tv1/tv1.tcp-fuse.vol     //client volume file

/var/lib/glusterd/vols/tv1/tv1.gfvm3.opt-volume_test-tv_1-b1.vol        //server volume file (2)

/var/lib/glusterd/vols/tv1/tv1.gfvm3.opt-volume_test-tv_1-b2.vol        //server volume file

/var/lib/glusterd/vols/tv1/tv1-rebalance.vol    //rebalance volume file (3)
===================================
(1)

/var/lib/glusterd/vols/tv1/trusted-tv1.tcp-fuse.vol
/var/lib/glusterd/vols/tv1/tv1.tcp-fuse.vol
 ==> both are quite similar except username and password present in trusted.


volume tv1-client-0
    type protocol/client
    option send-gids true
    option password c841565a-f1fc-4ffb-bb1c-2f5a0939c434
    option username f15ceb70-85fd-4be6-a084-757131d3bb0a
    option transport-type tcp
    option remote-subvolume /opt/volume_test/tv_1/b1
    option remote-host gfvm3
    option ping-timeout 42
end-volume

volume tv1-client-1
    type protocol/client
    option send-gids true
    option password c841565a-f1fc-4ffb-bb1c-2f5a0939c434
    option username f15ceb70-85fd-4be6-a084-757131d3bb0a
    option transport-type tcp
    option remote-subvolume /opt/volume_test/tv_1/b2
    option remote-host gfvm3
    option ping-timeout 42
end-volume

volume tv1-dht
    type cluster/distribute
    subvolumes tv1-client-0 tv1-client-1
end-volume

volume tv1-write-behind
    type performance/write-behind
    subvolumes tv1-dht
end-volume

volume tv1-read-ahead
    type performance/read-ahead
    subvolumes tv1-write-behind
end-volume

volume tv1-readdir-ahead
    type performance/readdir-ahead
    subvolumes tv1-read-ahead
end-volume

volume tv1-io-cache
    type performance/io-cache
    subvolumes tv1-readdir-ahead
end-volume

volume tv1-quick-read
    type performance/quick-read
    subvolumes tv1-io-cache
end-volume

volume tv1-open-behind
    type performance/open-behind
    subvolumes tv1-quick-read
end-volume

volume tv1-md-cache
    type performance/md-cache
    subvolumes tv1-open-behind
end-volume

volume tv1
    type debug/io-stats
    option count-fop-hits off
    option latency-measurement off
    option log-level INFO
    subvolumes tv1-md-cache
end-volume

-----------------------------------
(2) SERVER volume file

/var/lib/glusterd/vols/tv1/tv1.gfvm3.opt-volume_test-tv_1-b1.vol

[root@gfvm3 glusterfs]# cat /var/lib/glusterd/vols/tv1/tv1.gfvm3.opt-volume_test-tv_1-b1.vol
volume tv1-posix
    type storage/posix
    option volume-id 16a23423-2748-4541-9b17-787609bf72fb
    option directory /opt/volume_test/tv_1/b1
end-volume

volume tv1-trash
    type features/trash
    option trash-internal-op off
    option brick-path /opt/volume_test/tv_1/b1
    option trash-dir .trashcan
    subvolumes tv1-posix
end-volume

volume tv1-changetimerecorder
    type features/changetimerecorder
    option sql-db-wal-autocheckpoint 1000
    option sql-db-cachesize 1000
    option ctr-record-metadata-heat off
    option record-counters off
    option ctr-enabled off
    option record-entry on
    option ctr_lookupheal_inode_timeout 300
    option ctr_lookupheal_link_timeout 300
    option ctr_link_consistency off
    option record-exit off
    option db-path /opt/volume_test/tv_1/b1/.glusterfs/
    option db-name b1.db
    option hot-brick off
    option db-type sqlite3
    subvolumes tv1-trash
end-volume

volume tv1-changelog
    type features/changelog
    option changelog-barrier-timeout 120
    option changelog-dir /opt/volume_test/tv_1/b1/.glusterfs/changelogs
    option changelog-brick /opt/volume_test/tv_1/b1
    subvolumes tv1-changetimerecorder
end-volume

volume tv1-bitrot-stub
    type features/bitrot-stub
    option export /opt/volume_test/tv_1/b1
    subvolumes tv1-changelog
end-volume

volume tv1-access-control
    type features/access-control
    subvolumes tv1-bitrot-stub
end-volume

volume tv1-locks
    type features/locks
    subvolumes tv1-access-control
end-volume

volume tv1-upcall
    type features/upcall
    option cache-invalidation off
    subvolumes tv1-locks
end-volume

volume tv1-io-threads
    type performance/io-threads
    subvolumes tv1-upcall
end-volume

volume tv1-marker
    type features/marker
    option inode-quota off
    option quota off
    option gsync-force-xtime off
    option xtime off
    option quota-version 0
    option timestamp-file /var/lib/glusterd/vols/tv1/marker.tstamp
    option volume-uuid 16a23423-2748-4541-9b17-787609bf72fb
    subvolumes tv1-io-threads
end-volume

volume tv1-barrier
    type features/barrier
    option barrier-timeout 120
    option barrier disable
    subvolumes tv1-marker
end-volume

volume tv1-index
    type features/index
    option index-base /opt/volume_test/tv_1/b1/.glusterfs/indices
    subvolumes tv1-barrier
end-volume

volume tv1-quota
    type features/quota
    option deem-statfs off
    option timeout 0
    option server-quota off
    option volume-uuid tv1
    subvolumes tv1-index
end-volume

volume tv1-worm
    type features/worm
    option worm off
    subvolumes tv1-quota
end-volume

volume tv1-read-only
    type features/read-only
    option read-only off
    subvolumes tv1-worm
end-volume

volume /opt/volume_test/tv_1/b1
    type debug/io-stats
    option count-fop-hits off
    option latency-measurement off
    option log-level INFO
    subvolumes tv1-read-only
end-volume

volume tv1-server
    type protocol/server
    option auth.addr./opt/volume_test/tv_1/b1.allow *
    option auth.login.f15ceb70-85fd-4be6-a084-757131d3bb0a.password c841565a-f1fc-4ffb-bb1c-2f5a0939c434
    option auth.login./opt/volume_test/tv_1/b1.allow f15ceb70-85fd-4be6-a084-757131d3bb0a
    option transport-type tcp
    subvolumes /opt/volume_test/tv_1/b1
end-volume
[root@gfvm3 glusterfs]#

-----------------------------------
SERVER again
/var/lib/glusterd/vols/tv1/tv1.gfvm3.opt-volume_test-tv_1-b2.vol

-- Similar to above.
-----------------------------------
REBALANCE

/var/lib/glusterd/vols/tv1/tv1-rebalance.vol
[root@gfvm3 glusterfs]# cat /var/lib/glusterd/vols/tv1/tv1-rebalance.vol
volume tv1-client-0
    type protocol/client
    option password c841565a-f1fc-4ffb-bb1c-2f5a0939c434
    option username f15ceb70-85fd-4be6-a084-757131d3bb0a
    option transport-type tcp
    option remote-subvolume /opt/volume_test/tv_1/b1
    option remote-host gfvm3
    option ping-timeout 42
end-volume

volume tv1-client-1
    type protocol/client
    option password c841565a-f1fc-4ffb-bb1c-2f5a0939c434
    option username f15ceb70-85fd-4be6-a084-757131d3bb0a
    option transport-type tcp
    option remote-subvolume /opt/volume_test/tv_1/b2
    option remote-host gfvm3
    option ping-timeout 42
end-volume

volume tv1-dht
    type cluster/distribute
    subvolumes tv1-client-0 tv1-client-1
end-volume

volume tv1
    type debug/io-stats
    option count-fop-hits off
    option latency-measurement off
    option log-level INFO
    subvolumes tv1-dht
end-volume
[root@gfvm3 glusterfs]#
-----------------------------------
-----------------------------------------------------------------------
[root@gfvm3 glusterfs]# gluster volume info tv1

Volume Name: tv1
Type: Distribute
Volume ID: 16a23423-2748-4541-9b17-787609bf72fb
Status: Started
Number of Bricks: 2
Transport-type: tcp
Bricks:
Brick1: gfvm3:/opt/volume_test/tv_1/b1
Brick2: gfvm3:/opt/volume_test/tv_1/b2
Options Reconfigured:
performance.readdir-ahead: on
[root@gfvm3 glusterfs]#
-----------------------------------------------------------------------
[root@gfvm3 glusterfs]#
[root@gfvm3 glusterfs]# gluster volume status tv1
Status of volume: tv1
Gluster process                             TCP Port  RDMA Port  Online  Pid
------------------------------------------------------------------------------
Brick gfvm3:/opt/volume_test/tv_1/b1        49152     0          Y       1733
Brick gfvm3:/opt/volume_test/tv_1/b2        49153     0          Y       1753
NFS Server on localhost                     N/A       N/A        N       N/A 

Task Status of Volume tv1
------------------------------------------------------------------------------
There are no active volume tasks

-----------------------------------------------------------------------
[root@gfvm3 glusterfs]# mount | grep tv1
gfvm3:/tv1 on /mnt/master type fuse.glusterfs (rw,relatime,user_id=0,group_id=0,default_permissions,allow_other,max_read=131072)
[root@gfvm3 glusterfs]#
-------------------------------------------------------------------------
[root@gfvm3 ~]# cat /usr/local/etc/glusterfs/glusterd.vol
volume management
    type mgmt/glusterd
    option working-directory /var/lib/glusterd
    option transport-type socket,rdma
    option transport.socket.keepalive-time 10
    option transport.socket.keepalive-interval 2
    option transport.socket.read-fail-log off
    option ping-timeout 0
    option event-threads 1
#   option base-port 49152
end-volume
[root@gfvm3 ~]#
-------------------------------------------------------------------------------



performance measurement in gluster

===========================================

Performance measurement  - getting write statistics

The following steps should help you with profile info:

1) gluster volume profile start
2) Perform copy operations
3) gluster volume profile info (you will get stats of the FOPs at this point)
4) gluster volume profile stop

Please follow steps 1 through 4 twice. 1. Once for copy into glusterfs and
2. once for copy within.

===========================================