Amazon SES IP addresses

 Sometimes we may need to know SES IP address to mark them as whitelisted.

 

SES will maintain set of ipaddress to send emails. To know what are all those IP address

We need to run the following command in any Linux command prompt

dig TXT amazonses.com +short| grep 'v=spf1'

 

Once we execute will get following output

v=spf1 ip4:199.255.192.0/22 ip4:199.127.232.0/22 ip4:54.240.0.0/18 ip4:69.169.224.0/20 ip4:23.249.208.0/20 ip4:23.251.224.0/19 ip4:76.223.176.0/20 ip4:54.240.64.0/19 ip4:54.240.96.0/19 ip4:52.82.172.0/22 -all

 

These IP ranges are used to send email. This may change, it is better to add SPF record to whitelist otherwise we keep on checking if there is any change, we need to update our IP list

 

Windows we need to use following command

nslookup -type=TXT amazonses.com | find "v=spf1"

Allow access S3 bucket to specific VPC endpoint and Ip address

 

Even though you make it S3 bucket is private, if someone knows the access key,  he can access S3 bucket form anywhere using access key

 

If you want to restrict access to only to corporate network is possible in two ways

 

1.       Allow access to specific VPC end point, So that it will allow to access bucket once you part of specific VPC

2.       Allow access to specific IP Address

 

The above options will give addition level of security.

 

Allowing access to specific VPC end point

 To write S3 policy you use policy editor or Notepad.

Step 1: Change the relevant places in the following XML and copy

{

    "Version": "2012-10-17",

    "Id": "<<policy Name>>",

    "Statement": [

        {

            "Sid": "<<Any id>>",

            "Effect": "Deny",

            "Principal": "*",

            "Action": "s3:*",

            "Resource": [

                "arn:aws:s3:::<<bucket Name>>",

                "arn:aws:s3:::<<bucket Name>>/*"

            ],

            "Condition": {

                "StringNotEquals": {

                    "aws:SourceVpce": "<<VPC ID>>"

                }

            }

        }

    ]

}

 

Step 2: Go to the S3 bucket in which you want to restrict

Step 3: Go to the permission tab

Step 4: Past the copied policy in “Bucket policy” Section and save

 

Allowing access to specific IPS

 

{

    "Version": "2012-10-17",

    "Id": "<<Policy Id>>",

    "Statement": [

        {

            "Sid": "<<Policy ID>>",

            "Effect": "Deny",

            "Principal": "*",

            "Action": "s3:*",

            "Resource": [

                "arn:aws:s3:::<<Bucket Name>>",

                "arn:aws:s3:::<<Bucket Name>>",/*"

            ],

            "Condition": {

                "NotIpAddress": {

                    "aws:SourceIp": [

                       <<multiple Ips with comma separate>>

                    ]

                }

               

            }

        }

    ]

}

 

 

Note: if anything goes wrong you will not have access to bucket, you can remove policy only via root user access. Please careful before you do the change  

The table '/tmp/mysql/#sql_xxxxx' is full in MYSQL

 The table '/tmp/mysql/#sql_xxxxx' is full

 

The above-mentioned error may occur few different scenario.

We faced this issue suddenly when we updated our database from 8.0.21 to 8.0.23, After that when run a large query sometimes we started facing issue this.

 

Following discussion thread helped us to resolve this issue

https://bugs.mysql.com/bug.php?id=99100

 

Look like it is bug,

Based on this discussion in the forum,  we found a solution changing following parameter in MYSQL config helped us.

internal_tmp_mem_storage_engine=Memory;

 

 

 

S3 Bucket SNS Event Configuration: Unable to validate the following destination configurations.

 

Before Amazon S3 publish messages to a destination, you must grant the Amazon S3 principal the necessary permissions to call the relevant API to publish messages to an SNS topic

While creating SNS topics, we should have to provide a publish permission to S3 buckets

Step 1 : Create SNS topic ( Select standard, currently standard is supported by s3 notification)

Step 2: Access policy option select advanced option and update the JSON as like below.

In the JSON Editor change JSON below

{

 "Version": "2012-10-17",

 "Id": "example-ID",

 "Statement": [

  {

   "Sid": "example-statement-ID",

   "Effect": "Allow",

   "Principal": {

    "Service": "s3.amazonaws.com" 

   },

   "Action": [

    "SNS:Publish"

   ],

   "Resource": "SNS-topic-ARN",

   "Condition": {

      "ArnLike": { "aws:SourceArn": "arn:aws:s3:*:*:bucket-name" },

      "StringEquals": { "aws:SourceAccount": "bucket-owner-account-id" }

   }

  }

 ]

}

 

Step3: Create the topic and subscribe the topics.

Step 4: Go the S3 bucket and create event notification.

Step5: Select SNS option, select created SNS topic  and save changes

Restrict IAM user to access specific s3 bucket or folder

 

Sometime organization will have a separate s3 folder for each developer to manage their document. In that case system admin has to restrict the developer to specific bucket. There two way we can achieve this  

 

1. Create a separate policy for each user

2. Create bucket name same as IAM user name and write a one policy to restrict all

 

Create a separate policy for each user

Step1: Go the IAM and select user

Step2:  Click Add User and provide user name, Access type and necessary details then click Next permission

Step3: Select attach policy from permission screen and click Create Policy button

Step4: In the create policy screen select JSON and copy paste following JSON and change the bucket name

{

"Version": "2012-10-17",

"Statement": [

{

"Sid": "VisualEditor0",

"Effect": "Allow",

"Action": [

"s3:ListBucketMultipartUploads",

"s3:ListBucketVersions",

"s3:ListBucket",

"s3:ListMultipartUploadParts"

],

"Resource": "arn:aws:s3:::<<Bucket Name>>"

},

{

"Sid": "VisualEditor1",

"Effect": "Allow",

"Action": [

"s3:ListAllMyBuckets",

"s3:ListJobs"

],

"Resource": "*"

},

{

"Sid": "VisualEditor2",

"Effect": "Allow",

"Action": [

"s3:PutObject",

"s3:GetObject"

],

"Resource": "arn:aws:s3:::<<Bucket Name>>/*"

}

]

}

Step5: Click Review policy, enter policy name, description and click on create policy

Step6:  Click Refresh button, search and select created policy

Step 7: Click next and complete the action

 

Create folder name same as IAM user name and write a one policy to restrict all

Step1: In the IAM create a new policy with following policy

 

{

    "Version": "2012-10-17",

    "Statement": [

        {

            "Sid": "VisualEditor0",

            "Effect": "Allow",

            "Action": [

                "s3:ListBucketMultipartUploads",

                "s3:ListBucketVersions",

                "s3:ListBucket",

                "s3:ListMultipartUploadParts"

            ],

            "Resource": "arn:aws:s3:::${aws:username}"

        },

        {

            "Sid": "VisualEditor1",

            "Effect": "Allow",

            "Action": [

                "s3:ListAllMyBuckets",

                "s3:ListJobs"

            ],

            "Resource": "*"

        },

        {

            "Sid": "VisualEditor2",

            "Effect": "Allow",

            "Action": [

                "s3:PutObject",

                "s3:GetObject"

            ],

            "Resource": "arn:aws:s3:::${aws:username}/*"

        }

    ]

}

Step2: Attach this created policy against each user.

Launching EC2 instance from EBS snapshot in AWS

 

As part of backup process, we might have scheduled a EBS volume snapshot. Sometimes we will get a requirement to launch an instance from snapshot for various purpose.

The following steps to launch the EC2 instance from EBS snapshot.

Step1 : Select the snapshot to launch an EC2 instance

Step2: Create an AMI From selected snapshot

Step3: Launch instance from AMI.

Details steps as follows

1. Go to the EBS snapshot section

2. Select the snapshot

3. In the auction dropdown button select create image.

4. In the create image screen keep the default selection as it is. Enter the image name and description click create image to create an image.

 

5. Once image is created, Go to the AMIs section. Select the AMI from the Action drop down button click launch

 

6. In the launch instance screen select instance type, configure instance and configure security group to launch the new EC2 instance

Encrypt existing unencrypted MYSQL RDS instance in AWS

Currently AWS does not support to modify existing unencrypted Amazon RDS DB instance to encrypt the instance. Also, it does not support to create an encrypted read replica from an unencrypted instance.

To Modify existing unencrypted Amazon RDS DB instance to encrypt instance we have follow the following steps.

 

Step1 : Take the snapshot of existing unencrypted RDS instance

Step2: Convert unencrypted snapshot to encrypted snapshot

Step3: Restore the new RDS MYSQL instance from encrypted snapshot

Step4: Switch your application connections to new database

 

Before doing this, we must plan a proper downtime if we do a live database. We need to make sure as part of this process no transaction needs to be performed in the existing RDS instance otherwise we will have data loss.

 

If we have to minimize the downtime, we need to create a read replica to perform this step.

 

Step1 : Take the snapshot of existing unencrypted RDS instance

 1. In the database base choose the database

 2. Select the Action dropdown button and click Take Snapshot

3. Enter the snapshot name and click Take Snapshot button

 

Step2: Convert unencrypted snapshot to encrypted snapshot

1.       Go the snapshot and select the recent snapshot

2.       In the action drop down button select Copy Snapshot

3.       Enter the new snapshot name, region, select encrypt key and Copy snapshot

 

Step3: Restore the new RDS MYSQL instance from encrypted snapshot

1.      1. Select the encrypted snapshot

2.       2.  From the action drop down button select Restore Snapshot

3.       3. In the restore snapshot screen, enter new database name, VPC, Security group and Restore DB instance

Step4: Switch your application connections to new database