MongoDB Backups scripts and reference urls

How To Set Up Scheduled MongoDB Backups to DigitalOcean Spaces

Reference URLs

URL: : https://www.digitalocean.com/community/tutorials/how-to-set-up-scheduled-logical-mongodb-backups-to-digitalocean-spaces

URL: https://tecadmin.net/shell-script-backup-mongodb-database/

URL: https://sqlbak.com/blog/how-to-automate-mongodb-database-backups-in-linux

URL: https://tecadmin.net/shell-script-backup-mongodb-database/

URL: https://gist.github.com/sheharyarn/0f04c1ba18462cddaaf5

URL: https://simplebackups.io/blog/the-ultimate-mongodb-database-backup-script/

#!/bin/bash

######################################################################

##

##   MongoDB Database Backup Script 

##   Written By: Tushar Jadhav

##

######################################################################

export PATH=/bin:/usr/bin:/usr/local/bin

TODAY=`date +"%d%b%Y"`

######################################################################

DB_BACKUP_PATH='/backup/mongo'

MONGO_HOST='localhost'

MONGO_PORT='27017'

# If mongodb is protected with username password.

# Set AUTH_ENABLED to 1 

# and add MONGO_USER and MONGO_PASSWD values correctly

AUTH_ENABLED=0

MONGO_USER=''

MONGO_PASSWD=''

# Set DATABASE_NAMES to "ALL" to backup all databases.

# or specify databases names seprated with space to backup 

# specific databases only.

DATABASE_NAMES='ALL'

#DATABASE_NAMES='mydb db2 newdb'

## Number of days to keep local backup copy

BACKUP_RETAIN_DAYS=30   

######################################################################

mkdir -p ${DB_BACKUP_PATH}/${TODAY}

AUTH_PARAM=""

if [ ${AUTH_ENABLED} -eq 1 ]; then

 AUTH_PARAM=" --username ${MONGO_USER} --password ${MONGO_PASSWD} "

fi

if [ ${DATABASE_NAMES} = "ALL" ]; then

 echo "You have choose to backup all databases"

 mongodump --host ${MONGO_HOST} --port ${MONGO_PORT} ${AUTH_PARAM} --out ${DB_BACKUP_PATH}/${TODAY}/

else

 echo "Running backup for selected databases"

 for DB_NAME in ${DATABASE_NAMES}

 do

 mongodump --host ${MONGO_HOST} --port ${MONGO_PORT} --db ${DB_NAME} ${AUTH_PARAM} --out ${DB_BACKUP_PATH}/${TODAY}/

 done

fi

######## Remove backups older than {BACKUP_RETAIN_DAYS} days  ########

DBDELDATE=`date +"%d%b%Y" --date="${BACKUP_RETAIN_DAYS} days ago"`

if [ ! -z ${DB_BACKUP_PATH} ]; then

      cd ${DB_BACKUP_PATH}

      if [ ! -z ${DBDELDATE} ] && [ -d ${DBDELDATE} ]; then

            rm -rf ${DBDELDATE}

      fi

fi

######################### End of script ##############################

Setup Jenkins and Tomcat server

Login to aws instance or VM and add user

useradd devops

passwd devops

Edit this file /etc/ssh/sshd_config and enable PasswordAuthentication

PasswordAuthentication yes

Restart SSHD Service

systemctl restart sshd

------------------------------------------------------------------------------------------------------------

Install java 1.8 version

yum install java-1.8* -y

Set java path : /root/.bash_profile

export JAVA_HOME=/usr/lib/jvm/java-1.8.0-openjdk-1.8.0.191.b12-0.amzn2.x86_64

echo $JAVA_HOME

source /root/.bash_profile

------------------------------------------------------------------------------------------------------------

Download Jenkins repo

Ref : https://docs.aws.amazon.com/aws-technical-content/latest/jenkins-on-aws/installation.html

 

wget -O /etc/yum.repos.d/jenkins.repo http://pkg.jenkins.io/redhat/jenkins.repo

rpm --import https://pkg.jenkins.io/redhat/jenkins.io.key

yum install jenkins -y 

service jenkins start   or  systemctl start jenkin

Open 8080 port 

http://localhost:8080/

Password location :  /var/lib/jenkins/secrets/initialAdminPassword

 

Secure Jenkins server

1. Change admin password

2. Remove the secret password file ( /var/lib/jenkins/secrets/initialAdminPassword )

3. Configure Global Security - Enable Security

4. Configure Global Security - Enable Project-based Matrix Authorization Strategy and role base Authorization

5. Configure Global Security - Disable SSH Server

------------------------------------------------------------------------------------------------------------

Install Maven & Git  installation and configuration 

 

Ref : https://maven.apache.org/download.cgi

 

Create maven folder in opt

Install wget and downloads below maven packages

wget http://mirrors.estointernet.in/apache/maven/maven-3/3.6.0/binaries/apache-maven-3.6.0-bin.tar.gz

###  /root/.bash_profile  ######

 

# User specific environment and startup programs

export JAVA_HOME=/usr/lib/jvm/java-1.8.0-openjdk-1.8.0.191.b12-0.amzn2.x86_64

 

M2_HOME=/opt/maven/apache-maven-3.6.0

M2=$M2_HOME/bin

PATH=$PATH:$JAVA_HOME:$HOME/.local/bin:$HOME/bin

------------------------------------------------------------------------------------------------------------

Select maven_invoker and github

 

And path in Global Tool Configuration

------------------------------------------------------------------------------------------------------------

Install Publish Over SSH  plugging for Ansible integration and send build artifacts over SSH

Download below tomcat package

wget http://mirrors.estointernet.in/apache/tomcat/tomcat-8/v8.5.38/bin/apache-tomcat-8.5.38.tar.gz

 

Ref : https://tomcat.apache.org/download-80.cgi

 

cd /opt/apache-tomcat-8.5.38/bin/      -----  After extract you  get this folder

 

chmod +x startup.sh  shutdown.sh     ----- Give execute permission to  startup.sh and  shutdown.sh 

 

 

Create soft link 

ln -s /opt/apache-tomcat-8.5.38/bin/startup.sh /usr/bin/tomcatup

ln -s /opt/apache-tomcat-8.5.38/bin/shutdown.sh /usr/bin/tomcatdown

 

Tomcatup   -------------------------   start tomcat

 

 ps -ef |grep tomcat   -----------------------  check tomcat is running or not

 

/opt/apache-tomcat-8.5.38/conf   ----------   go to conf directory and edit server.xml 

 

Edit server.xml and change tomcat port

 

 

Change 8080 to 8090

 

Restart tomcat

 

tomcatdown   --------------  stop the service

 

tomcatup      ---------------   start the service

URL : http://ServerIP:8090/manager/

Solution for above issue: 

 find / -name context.xml

/opt/apache-tomcat-8.5.38/conf/context.xml

/opt/apache-tomcat-8.5.38/webapps/host-manager/META-INF/context.xml

/opt/apache-tomcat-8.5.38/webapps/manager/META-INF/context.xml

 

Edit both the files and uncomment below line

Restart tomcat

 

tomcatdown   --------------  stop the service

 

tomcatup      ---------------   start the service

How to set user and password  add following roles

 

Edit this file /opt/apache-tomcat-8.5.38/conf/tomcat-users.xml

 

<role rolename="manager-gui"/>

<role rolename="manager-script"/>

<role rolename="manager-jmx"/>

<role rolename="manager-status"/>

<user username="admin" password="admin" roles="manager-gui, manager-script, manager-jmx, manager-status"/>

<user username="deployer" password="deployer_user" roles="manager-script"/>

<user username="tomcat" password="s321cret" roles="manager-gui"/><feff>

---------------------------------------------------  END  -----------------------------------------------------

Unable to Retrieve Default Windows Administrator Password in AWS EC2

 Error: 

Password not available yet. Please wait at least 4 minutes after launching an instance before trying to retrieve the auto-generated password.

Note: Passwords are generated during the launch of Amazon Windows AMIs or custom AMIs that have been configured to enable this feature. Instances launched from a custom AMI without this feature enabled use the username and password of the AMI’s parent instance

 

Solutions:

The below steps are worked for me. 

1.Connect to your Windows instance and customize it. 

2.Search for and run the EC2LaunchSettings application if the windows is 2016
or later or if the windows server 2012 R2 and earlier version of Windows server 2012R2 open EC2Config service application.

3.Enable Random(Retrieve from console) 

4.Click Shutdown the system with Sysprep option.

5.Create AMI once instance got shutdown. 

6. Then launch the system using new Keypair.

 

How to install and configure sSMTP on CentOS

 sSMTP is alternative to Sendmail. which is send the mail from existing mail server like gmail, yahoo or own server. This program to send mail via standard or /usr/bin/mail useragents. it is used to send mail to external mail addr after authentication with account It's also use in nagios, install in nagios server and add the admin email address entry in Contact.cfg file.

1. First step : Install sSMTP Packages :

# yum -y install ssmtp

2. Now we will edit the ssmtp configuration file.

# cd /etc/ssmtp

Here is the custom configuration(here we use gmail account)

# vim /etc/ssmtp/ssmtp.conf

root=**********@gmail.com  
            ## Redirect mail for root@ to postmaster@

MailHub=smtp.gmail.com:587 ## SMTP server hostname and port
AuthUser=*****@gmail.com   ## Your DreamHost mailaccount
AuthPass=        ## The password for the mail account
UseSTARTTLS=YES            ## The password for the mail account

UseTLS=NO                  # Secure connection (SSL/TLS)
FromLineOverride=YES       # Force the From: line
Hostname=mail.gmail.com    # The name of this host
RewriteDomain=mail.gmail.com 
            # The host the mail appears to be coming from

## Compulsory field.

3. If you would like to change the “From“Display Name, edit the /etc/passwd and modify the user alias

# vim /etc/passwd 
user1:User1:507:507::/home/user1:/bin/bash

Note :Before we start using sSMTP need to stop the sendmail.

# service sendmail stop
# sudo chkconfig –levels 2345 sendmail off

4. Move or replace the sendmail and thencreate a symbolic link for sSMTP to sendmail with authentication.

# mv /usr/sbin/sendmail/usr/sbin/sendmail.ori
# ln -s /usr/sbin/ssmtp/usr/sbin/sendmail

5.Now it's time to test sSMTP.

# echo”Sending Test mail to external & Internal email address”|
 mail -s”this is the test msg using sSMTP” testssmtp@gmail.com

Fixing a corrupt /etc/sudoers file in Linux VM in Azure

I was editing the /etc/sudoers file with vim on a linux VM (RHEL 7.5) in Azure trying to remove or disable being prompted for a password every time I sudo.

I added the following to the file

root        ALL=(ALL:ALL) ALL
myadminuser     ALL=(ALL:ALL) ALL     NOPASSWD: ALL

Apparently that does not follow the correct syntax so immediately after I was not able to sudo. Below is the error meesage:

[myadminuser@MYSERVER ~]$ sudo -i
>>> /etc/sudoers: syntax error near line 23 <<<
sudo: parse error in /etc/sudoers near line 23
sudo: no valid sudoers sources found, quitting

sudo: unable to initialize policy plugin

Since on the Azure VMs you don't have the root password, then you're stuck as the regular user do not have permissions to edit the sudoers file and you can't sudo to root.

You could mount the VM disk to another VM and then edit the file that way, but that is cumbersome.

Fix:

From the Azure portal start Cloud CLI, choose Powershell

Run the following command to make /etc/sudoers editable by master

az vm run-command invoke --resource-group YOUR_RESOURCE_GROUP --name YOURVM --command-id RunShellScript --scripts "chmod 446 /etc/sudoers"

This gives the regular user permission to edit the file

with nano or VI undo the changes (i just deleted the NOPASSWD: ALL):

login to the server and edit sudoers file
nano /etc/sudoers (no sudo since you have access)

after edit, run the below command to configure default access to file.

az vm run-command invoke --resource-group YOUR_RESOURCE_GROUP --name YOURVM --command-id RunShellScript --scripts "chmod 440 /etc/sudoers"

I got fixed my problem after run above commands

For memory limit issue while uploading the product image in magento 1

For memory limit issue while uploading the product image



Go To lib/Varien/Image/Adapter open Gd2.php

Now search for method protected function _convertToByte($memoryValue) i.e

protected function _convertToByte($memoryValue)
{
    if (stripos($memoryValue, 'M') !== false) {
        return (int)$memoryValue * 1024 * 1024;
    }
    elseif (stripos($memoryValue, 'KB') !== false) {
        return (int)$memoryValue * 1024;
    }

   return (int)$memoryValue;
}
Make the following changes in the above method like :

protected function _convertToByte($memoryValue)
{
     /*
     if (stripos($memoryValue, 'M') !== false) {
         return (int)$memoryValue * 1024 * 1024;
     }
     elseif (stripos($memoryValue, 'KB') !== false) {
         return (int)$memoryValue * 1024;
     }

     return (int)$memoryValue;
     */
     return 2147483648;
     //1024*1024*1024*2 = 2G
}
Hope this will help someone :)

ELK installation

1). Introduction
This is an ELK used for monitoring all logs at one place.


2) Pre-requisite:
JAVA should be installed in order to setup ELK.
apt-get update
apt-get install default-jre
apt-get install default-jdk
add-apt-repository ppa:webupd8team/java
apt-get update
apt-get install oracle-java8-installer
#edit the file and add below line….vim /etc/environment
JAVA_HOME="/usr/lib/jvm/java-8-oracle"
source /etc/environment
echo $JAVA_HOME…This should reflect the path mentioned in the file.
java --version


3) Install Docker and Docker-compose apt-get install docker
apt-get install docker-compose
systemctl start docker


4) Git Clone the image git clone https://github.com/deviantony/docker-elk.git It will create a directory called docker-elk


5) Editing the .yml files If you are using custom ports then we will have to change the respective ports in the respective files as well as docker-compose.yml file.

Default ports:
9200 – Elastic search
5601- Kibana
5044- Log stash


6) Creating Containers cd docker-elk
docker-compose up --build -d
It will take time and you can verify if the containers are created
docker ps


7) Web-page
Hit the web-page

<ip>:9200—you must see the .jason format. This is the confirmation that elastic search is installed correctly.





<ip>:5601—you must see the kibana page.




8) Setup of Beat agent on Client Side Click on Logs à Add data à <type of logs you require> à Select the OS à Follow the instructions to install the agent.




9) Adding log paths
On client side after installing the Beat. Go to its directory to add the log paths
cd /etc/filebeat/modules.d
Edit the file vim nginx.yml
There are two sections of access and error. We can give path of the logs we want to fetch in kibana over here.



--------------------------------------------------------------------------------------------------