RSYNC COMMAND
rsync (remote sync) is used to transfer data between two servers. it is basically a remote synchronization tool which helps to copy or synced data from one machine to another machine. it is a Unix-Linux based tool.
BENEFITS OF RSYNC COMMAND:
Rsync is a very useful tool for system admins because every sysadmin should use this command at list once in a day. It has many benefits we will see some of them important points.
1. Rsync efficiently sync files to and from systems.
2. It is faster sync tool than any other syncing tools like SCP.
3. It has the feature of incremental data transfer means it will not sync data which is already available on another system.
4. Rsync has provided progress of data transfer which is also plus. you will know how much data is copied and how much is remaining.
5. when sending and receiving data rsync will use compression and decompression method. It comes with benefits of using less bandwidth.
6. Rsync will copy everything as it is which means sync links, owner, permission, groups.
SOME IMPORTANT OPTIONS OF RSYNC COMMAND:
-v, --verbose : Verbose or Providing detailed output for diagnostic purposes.
-a, --archive : archiving allow copying files with symlinks, permissions, owners, groups etc
-z, --compress : Compress data during file transfer
-h, --human-readable : output numbers in a human-readable format
-P, --progress : show progress during transfer
_____________________________________________________
WHAT IS FTP SERVER?
Ftp server stands for File transfer Protocol. Defination on ftp server explain everything in full form of ftp.
It is used for transferring files from one server to another via secure way.
Ftp is TCP based service and there is no UDP component for FTP.
FTP utilizes two port one is command port i.e. 21 and other is data port i.e. 20.
WHY WE NEED FTP SERVER?
FTP Server is basically using for data transfer. We need ftp to provide secure access on server codebase to developers or also for clients.
As i am working as linux admin lots of time client need ftp access to codebase directory on server to transfer some media files or to work on some code so that time ftp is very useful to provide access in secure way. Here secure way means we can use another port no. instead of default port no. which is easy to guess and also we can whitelist ftp port for Client office only or for specific public ip. This way ftp help to manage data transfer process very well.
____________________________________________________
NFS SERVER
NFS means Network File System which is using to share one system data or directory over the remote network on multiple systems and you will able to access shared directory on your local system as local files or directory.
WHY WE NEED NFS SERVER?
- NFS allows local access to remote files.
- with help of NFS, we can create centralized storage which will be accessible from multiple systems.
- No need to refresh filesystem to get new files.
- Not necessary that both the client and host machine has a same operating system
client :
mount -t nfs 192.168.3.39:/sharenfsdir /mnt/nfsshare
/etc/fstab ----add below entry in fstab
192.168.3.39:/sharenfsdir /mnt/nfsshare nfs defaults 0 0
SOME IMPORTANT COMMANDS FOR NFS :
- showmount -e : it will shows you available shared directory on server.
- showmount -d : List all the sub directories.
- exportfs -a : it will help to reflect and listed the changes done under the /etc/exports
- exportfs -v : Display's list of share files and options on server.
- exportfs -r : Reflect the changes after modifying /etc/exports
_________________________________________________
REDIS CACHE
Redis Cache is an open source, advanced key-value store and a serious solution for building high-performance, scalable web applications.
Redis cache will handle the persistent data. Redis is more efficient caching application because it consumes comparatively fewer memory resources for metadata.
The good thing about the Redis cache, it will supports Master-slave replication which means whenever once of your Redis node goes down other will handle the request. It is providing stronger data structure.
Redis will supports multiple databases using same server instance. With that much of feature Redis will obviously first choice of any PHP developer to improve performance of website.
FEATURES OF REDIS :
- Redis cache handle persistent data
- Redis cache is more efficient because it consumes comparatively fewer memory resources for metadata.
- Redis supports master-slave replication.
- Redis has stronger data structures.
- Redis key-length has a maximum of 2GB.
- Redis support for multiple databases using same server instance.
______________________________________________________
AWS AMI [AMAZON MACHINE IMAGE]
In the full form, you will understand the use of AMI. It's basically a copy of your running EC2 Machine which we store on AWS as a backup of your machine.
You can transfer AMI with specific AWS accounts without making the AMI public. All you need are the AWS account IDs.
____________________________________________________
AWS CLOUD ELASITC COMPUTE CLOUD (EC2)
Aws Cloud provides very user-friendly console by which you can launch your own EC2 instance without any hard work.
Aws Cloud provides more than 20 instance types and you can use whichever is good for your project and according to your budget.
_______________________________________________________
INTRODUCTION MEMCACHED
You will get an idea about the working of Memcached with his name only.
Memcached is an open source high performance and free caching system or service.
It's helps to cache API calls, database data and pages in RAM to perform application faster.
This cached system works on TCP protocol which means you can able to install it on a separate server and use it on another server which leads to the big hash table to store your data.
One more important thing about Memcached is if your Memcached is running out of space then older data is overwritten or replaced by newer data. It will work as a non-persistent cache.
It will work or runs on default port i.e. 11211
You all know the open source applications are totally free cost. Memcached is also Open source so you do not need to pay anything to make your site perform better.
You can use Memcached to decrease the load on the database by caching database queries which are most frequently or more user visited.
ADVANTAGES:
1. It is open source cache system which means its freely available.
2. It will work as cross-platform.
3. It will also handle the sessions.
4. it is a distributed memory system.
DISADVANTAGES:
1. Data will not be persistent means data will be flush out if you restart the service.
2. It will be using only 1/4 of the size of your RAM memory, so if your size out of space then data is overwritten.
WHEN WE USE MEMCACHED?
We will use memcached to decrease the load on the database by caching database queries which are most frequently or more user visited.
RESTART SERVICE
/etc/init.d/memcached start|stop|restart
CHECK MEMCACHED IS RUNNING
netstat -tulnp | grep memcached
tcp 0 0 127.0.0.1:11211 0.0.0.0:* LISTEN 1391/memcached
______________________________________________________
VARNISH CACHE
Varnish cache is one of the best caching application which you can use to optimize PHP application. It is the latest and most used stable caching application. It will lead to your site to perform better and smooth.
Varnish cache is a web application accelerator also known as HTTP reverse proxy.
It is working on port 80 as a front web application. Varnish will improve your website speed 1000x faster and it's work well in good server architecture.
Varnish always run on 80 and 443 [if SSL is configured]port because on internet domains run through the HTTP or HTTPS. To configure varnish cache you need to understand the working of varnish vcl conf file.
Varnish cache will support multiple platforms like Linux, FreeBSD and various Solaris-descendants like Oracle Solaris, OmniOS, and SmartOS.
If you really want your site to perform better then varnish cache is the best option for your website. It has lots of benefits when it comes to providing performance for the website.
_________________________________________________________
LVM [ LOGICAL VOLUME MANAGER ]
LVM is a tool for logical volume management which is used to allocating disks and resizing logical volumes.
With the help of Logical Volume Manager, you will be able to increase and decrease the size of your disks or hard drives and also allocate it to one or more physical volumes.
WHY WE NEED LVM?
LVM is very useful in a various scenario where your hard disk is getting full, and you want to increase hard drive size without losing data, or also you can reduce as per the requirement.
COMMANDS USE IN LVM
$ pvcreate: - is used to create a physical volume on the harddrive.
$ pvdisplay :- is used to display created a physical volume on the harddisk.
$ pvremove :- is used to remove a physical volume from harddisk.
$ vgcreate :- is used to create volume group by adding created physical volume in a single group.
$ vgextend :- we can also extend the already created volume group with the help of vgextend command.
$ vgdisplay :- is used to display volume groups on harddrives.
$ vgremove :- we can also remove created volume groups from harddrives with the help of vgremove command.
$ lvcreate :- with the support of this command, we can create logical volume as per the user required a size from the Volume group.
$ lvdisplay :- is used to display created volumes.
$ lvreduce :- we can also reduce the size of already created logical volumes with the help of lvreduce command without losing data.
$ lvremove :- we can also remove created logical volumes if we don’t require.
$ lvextend :- is used to extend the size of the already created logical volume.
REDUCE VOLUME SIZE
if you want to reduce the size of Logical volume use below commands,
umount /dev/mynewvg/lvol1
e2fsck -f /dev/mynewvg/lvol1
resize2fs /dev/mynewvg/lvol1 600M
lvreduce -L -500M /dev/mynewvg/lvol1
EXTEND VOLUME GROUP
if you want to increase the Volume group, you need to add another physical volume in volume group use below command,
# vgextend /dev/mynewvg /dev/sda4
Extending logical volume lvol1 to 3.99 GiB
Logical volume mynewvg successfully resized
________________________________________________________
SWAP PARTITION:
Swap is extra space which is used when the physical memory (RAM) running out of space or full. When the system needs extra memory and the RAM is full that time inactive pages in RAM memory will move to the swap storage.
You can also call Swap as another small RAM which helps to boost performance while physical ram running out of space.
BENEFITS OF SWAP MEMORY:
- Provide extra space when physical memory full
- Move not-so-needed items from high-speed memory
- Allow to hibernate which means without swap hibernation is not possible on Linux.
DISADVANTAGES:
- Consume disk space on server
- Not necessarily improve performance
___________________________________________________________
MONGODB:
- MongoDb is an open-source application. It will be designed for ease of development.
- MonogDb is a NoSql database which means you can not store data in tables like Mysql/Postgres.
- MongoDb stores data in JSON format in the document structure.
- MongoDb available in only 64-bit for Debian/mint/ubuntu.
- MongoDb runs on default port "27017".
- MongoDb main configuration file is "mongod.conf" which is resides under "/etc" directory.
KEY FEATURES:
High Performance
- Index support faster queries
- use keys from Arrays
- Reduce Input/Output activity on database
Rich Query Language
- Support faster read & write operations
High Availability
- Replication feature provides automatic failover & data redundancy
Horizontal Scalability
- Supports sharding of data over cluster system's
Support for Multiple Storage Engines
- WiredTiger Storage Engine
- MMAPv1 Storage Engine.
________________________________________________________
TAR COMMAND
The Linux “tar command” stands for tape archive, which is used to deal with the tape drive backups.
The tar command used to rip a collection of files and directories into highly compressed archive file commonly called tarball or tar, gzip and bzip in Linux.
tar files easily moved from one disk to another or machine to machine.
The GNU tar command included with Linux distributions has integrated compression. It can create a .tar archive and then compress it with gzip or bzip2 compression in a single command. That’s why the resulting file is a .tar.gz file or .tar.bz2 file.
- c – Creates a new .tar archive file
- v – Verbosely show the .tar file progress
- f – File name type of the archive file.
- t - list the contents of .tar file
- x - untar or extract the files.
- C (capital) - used to extract the files in another directory or folder.
- z - used to compress tar with .gz extention
- j – filter archive through bzip2.
- r – append or update files or directories to existing archive file.
- W – Verify a archive file.
- wildcards – Specify patters in unix tar command.
__________________________________________________________
ELASTICSEARCH :
Elasticsearch is a distributed, open source search and analytics engine, designed for horizontal scalability, reliability, and easy management. elasticsearch combines the speed of search with the power of analytics via a sophisticated, developer-friendly query language covering structured, unstructured, and time-series data.
FEATURES OF ELASTICSEARCH :
- Real-Time Data
- Real-Time Advanced Analytics
- Massively Distributed
- High Availability
- Multitenancy
- Full-Text Search
- Document-Oriented
- Schema-Free
- Developer-Friendly, RESTful API
- Per-Operation Persistence
_________________________________________________________
ARCHIVE :
Basically it means you take 10 files and combine them into one file.
Archive does not reduce the size of files when its combined.
For eg if you have 10 files with 100kb per size after archiving these 10 files size of single combined file is 1000kb.
ZIP
Zip is a compress tool which is available in most of the operating systems such as Linux/Unix, Apple OS, Microsoft OS etc. In this post we will see how to install, use and tips about zip command.
Zip has a feature to provide speed for compression process below are some points which would help you to understand zip compression process clearly.
To regulate the speed of compression using value from 0 to 9 :
- 0 indicates no compression just store or archive files.
- 1 indicates fastest compression speed but less compression.
- 9 indicatest slowest compression speed but more compression.
- 6 is the default compression speed.
ZCAT
Sometime we have file with .gz compression and we want to read content of that file without extracting gz file so in this situation Zcat command would read the file content without extracting and also used for extract purpose.
BZCAT
bzcat also work like
zcat used to extract or to read the contents of file
________________________________________________________
UMASK ON LINUX:
Umask stands for
User mask or
User creation mask. When you create any new file or directory on linux system. These files get default permissions from system. This permission is provided or given by Umask configuration on linux. In simple words, it is a system default permissions for new files or directories.
DEFAULT UMASK VALUE :
By default umask values define under the "/etc/profile" or "~/.bashrc"file. On most of the linux distros default value set to files "022".
For directory, default minimum and maximum value is "000" & "777"
For files, default minimum and maximum value is "000" & "666"
So, with a umask of 022, the default permissions for a file will become 644 (rw-r--r--, 666-022) and the default permissions for a directory will become 755 (rwx-r-xr-x, 777-022).
BELOW ARE THE COMMON UMASK VALUES:
$ 000 : Full Access to Everyone or 666
$ 006 : No Access to Other or 660
$ 022 : Full Access to Owner and Read to Group & Other or 644
$ 066 : Full Access to Owner and No access to group & others or 600
Normally, you can subtract from 666 but be very careful as it may be 777. Before changing or working on umask make sure what umask value is set on your linux distro.
____________________________________________________
FSTAB [FILE SYSTEM TABLE] :
FSTAB is a file which contains all the information about partitions and storage devices present on the system. This file is basically located under the "/etc/" directory.
This file gives you information of where your storage devices should be mounted.
FSTAB is very critical and important file present in "/etc" directory where all the configuration files stored. It is the responsibility of SysAdmin to properly create and maintain this file.
You need to understand the structure of this file before writing anything into this. Because if add something with wrong format or structure it will result in crashing your system. You need to handle this very carefully.
Total six columns are present in FSTAB file. Each column defines or performs a different role. To add or mount new device use new row.
LET'S UNDERSTAND THE WORKING OF EACH COLUMN.
1ST COLUMN:
The first column will define the "Label" of partitions. For eg. "LABEL=/boot" or driver's path, eg. "/dev/cdrom". Device driver's path tells the system to mount the device with the mentioned device driver.
2ND COLUMN:
The second column describes the mount point for the filesystem. There is some filesystem should be specified as 'none' like swap partition. The mount point is actually a name of the directory where that device is mounted. Using this mount point we can be able to view and modify the content of that partition. You can modify the mount point according to your requirement.
3RD COLUMN:
The third column will be used to define the Filesystem type of partition or device. Several no. of filesystems supported by Linux and some of them listed below,
- ext2
- ext3
- iso9660
- autofs
- nfs
- swap
If you are not sure about the filesystem then use "auto" option. "auto" will help to determine the file system and mount the device with that filesystem.
4TH COLUMN:
The fourth column is for permissions to be given to the partition at the time of booting. There are many options which constitute the fourth column.
They are as follows : -
1) ro - Read Only
2) rw - Read Write
3) auto - Mount on startup
4) noauto- Do not mount on startup
5) user - Any user can mount, but only unmount device mounted by him
6) nouser- the Only root can mount & unmount the device
7) users - Every user can mount and also unmount the device mounted by others
8) owner - Same as user (above no. 5)
9) dev - User can use device driver to mount the device
10) nodev - User cannot use device driver to mount the device
11) exec - Users can execute binaries on the partition
12) noexec- Users cannot execute binaries on the partition
13) async - Asynchronous, whenever a file is saved it will be first saved in the RAM and after 30 seconds all the queued files will be written on the hard disk
14) sync - Synchronous, whenever a file is saved it will be directly written to the hard disk
15) suid - Allow set-user-identifier for the device where users are allowed to run binaries even though they do not have to execute permissions. These binaries are temporarily made available to them to perform certain tasks
16) nosuid- Do not allow set-user-identifier
17) defaults- auto, rw, dev, async, suid, exec & nouser
5TH COLUMN:
This column is used as a backup option. It will contain two values either 0 or 1. Here 0 stands for "no" and 1 stands for "yes". if option 1 used then system checks are enabled at the time of booting and if 0 used then system checks will be ignored. Backup option supported only ext3 filesystem which means for another filesystem this option should be disabled by default.
6TH COLUMN :
6th column is using for "fsck" option. Fsck stands for filesystem check. with this option system will scan the filesystem at time of start up. Here The / partition is assigned top priority i.e. 1 and the rest of the partitions are assigned second priority i.e. 2. If the value is set to 0 means no scanning will be done at the time of startup. If the same number is given to different partitions then the partitions are scanned together with equal priority. This minimizes error because if a link is present on one partition with higher priority and the source file in another partition with a priority lower than the link, it will give an error.
________________________________________________________
RPM PACKAGE MANAGER:
RPM stands for Red hat Package Manager. It is a powerful package manager tool for Redhat, open-suse, fedora etc. RPM can be used to install, build, verify, update, query and remove individual application or software packages.
RPM is default package manager for RedHat Linux systems.
RPM are basically files which contain installable software and that package has .rpm extension or suffix.
RPM is a free tool created or developed by Redhat software.
RPM will store information about packages in a database "/var/lib/rpm".
ADVANTAGES OF RPM PACKAGE MANAGER:
- Straightforward program installation and uninstallation
- Automatic installation
- Ease of updating programs originally installed with RPM
- Availability of versions for most distributions of Linux
DISADVANTAGES OF RPM PACKAGE MANAGER:
- Distribution packages
- Architecture-specific
- Dependency
_______________________________________________________
1. SYMLINK [SYMBOLIC LINK]:
A symlink is also called as Soft link. Symlink is a file which contains a reference to another file or directory in the form of absolute or relative path.
In simple word, you can create a shortcut of file or directory to the other path using symlink feature.
IMPORTANT POINTS ABOUT THE SYMLINK:
$ Links have different inode numbers.
$ Removing symlink will not affect the original file but if you remove original file then symlink will not work. the symlink will be changed to red color once original file removed
$ Symbolic links are set up using the ln command with the -s option.
$ Symlink is slightly slower at runtime compared to hard link but it's more flexible and more often used in day to day admin work.
$ Symlink always created with the 777 permission.
$ Symlink can create directories
HARDLINK :
A HardLink is where a file has two names which are both on an equal weighting, and both of the file names in the "inode table" point directly to the blocks on the disc that contain the data.
IMPORTANT POINTS FOR HARDLINK :
$ All Links have same inode number.
$ Links have actual file contents.
$ Removing any link just reduces the link count but doesn't affect other links.
$ Hard links can not cross partition which means you can not create hard link of one file to another partition.
$ Hardlinks can not create directories.
$ Hardlink can be create using ln command without any option.
$ Try creating hardlink with directories it will give you the error like "hard link not allowed for directory".
$Try creating hardlink to another partition it will produce an error like " failed to create hard link ‘/boot/qwe’ => ‘/opt/new-file/abc’: Invalid cross-device link".
_____________________________________________________
1. ABOUT SCREEN COMMAND
screen command is used to run the multiple processes in different shells as a background process.
It is a very useful command for system admins. Screen will offers multiple terminal processes inside a one single terminal manager.
2. BENEFITS OF SCREEN COMMAND
- screen command will help to create a process which actually runs in the background and if you are network got fluctuated and if you are running any important command then normal window gets disconnected but in the screen, you can again login on the machine and reattached screen with running process.
- It helps sysadmins to run rsync command with big data transfer, restoring or dumping large MySQL file etc this time screen command plays a very important part.
____________________________________________________
1. ABOUT SSL CERTIFICATE
SSL stands for Secure Socket Layer.
SSL is basically used to encrypt the site information and make a secure connection. It is maintaining encrypted link or secure way of connection between a Server and Client.
SSL helps to maintain or allow sensitive information like Credit Card nos., Usernames, Passwords, emails etc.
SSL will not allow or stop hackers to stolen important information from websites. It will provide a guarantee to customers that the important data will be safe with or because of SSL.
2. WHAT IS SELF-SIGNED SSL CERTIFICATE?
We just see the information about SSL Certificate. Now understand about the
Self-signed SSL certificate.
Self-Signed Certificate is basically a certificate which is created or signed by a person with local encryption technique.
This type of certificate is not a tested or approved by
trusted certificate authority. Self Signed certificates are not used on live websites.
If any one of the people using self-signed then customers will get the message "
The Site Security Certificate not Trusted!".
No one is using a Self-signed certificate on live sites.
3. WHEN SHOULD A SELF-SIGNED CERTIFICATE AND SHOULDN'T BE USED?
The self-signed certificate should not be used on Live websites like where website using credit cards info, username, passwords etc. if you used it then you are site going under risk of hack information. Hackers will steal or tampered the important information from your website. Also, you are customers will see the error message while accessing your site. Customers will not go to provide any sensitive information with the non-trusted site. It will definitely harm your business. So don't use any kind of Self-signed certificate on your live website.
Self-signed certificate is basically used on a development server where developer's need to test or develop some functionality or application. You no need to pay for trusted certificate when you want to implement it on local or development server. This is the main importance of self-signed certificate.
