Skip to content

Email Validation script needed and verifies deliverability without emailing

0

Hi AWS, I have a large database of email addresses that I need to verify deliverability without emailing and I want to develop the script and run it on AWS. The uploaded email verification script is in PERL but I think it will be better to use Python or Golang language.

#!/usr/bin/perl
use strict;
use warnings;
use Net::DNS;
use IO::Socket::INET;

# Function to check MX records and validate email
sub validate_email {
    my ($email) = @_;

    # Split the email into local part and domain
    my ($local_part, $domain) = split(/@/, $email);

    # Validate email format
    unless ($local_part && $domain) {
        die "Invalid email format\n";
    }

    # Get MX records for the domain
    my $resolver = Net::DNS::Resolver->new;
    my @mx_records;
    my $query = $resolver->query($domain, 'MX');

    if ($query) {
        foreach my $rr ($query->answer) {
            if ($rr->type eq 'MX') {
                push @mx_records, { exchange => $rr->exchange, preference => $rr->preference };
            }
        }
    } else {
        die "Failed to resolve MX records for domain $domain\n";
    }

    # Sort MX records by preference
    @mx_records = sort { $a->{preference} <=> $b->{preference} } @mx_records;

    if (!@mx_records) {
        die "No MX records found for domain $domain\n";
    }

    # Connect to the first MX server
    my $mx_server = $mx_records[0]->{exchange};
    my $socket = IO::Socket::INET->new(
        PeerAddr => $mx_server,
        PeerPort => 25,
        Proto    => 'tcp',
        Timeout  => 30,
    );

    unless ($socket) {
        die "Failed to connect to MX server $mx_server\n";
    }

    # Communicate with the server
    my $response = <$socket>;
    unless ($response =~ /^220/) {
        die "Unexpected response from server: $response\n";
    }

    # Send HELO
    print $socket "HELO example.com\r\n";
    $response = <$socket>;
    unless ($response =~ /^250/) {
        die "Unexpected response to HELO: $response\n";
    }

    # Send MAIL FROM
    print $socket "MAIL FROM:<test\@example.com>\r\n";
    $response = <$socket>;
    unless ($response =~ /^250/) {
        die "Unexpected response to MAIL FROM: $response\n";
    }

    # Send RCPT TO
    print $socket "RCPT TO:<$email>\r\n";
    $response = <$socket>;

    # Check if the email address is accepted
    if ($response =~ /^250/) {
        print "Email address $email is valid\n";
    } elsif ($response =~ /^550/) {
        print "Email address $email is invalid\n";
    } else {
        print "Unexpected response to RCPT TO: $response\n";
    }

    # Send QUIT
    print $socket "QUIT\r\n";
    close $socket;
}

# Main program
if (@ARGV != 1) {
    print "Usage: $0 <email>\n";
    exit 1;
}

my $email = $ARGV[0];
validate_email($email);

The python equivalent of the script:

import re
import dns.resolver
import concurrent.futures
import random
import requests

# List of proxies for IP rotation
proxies = [
{'http': 'http://IP_ADDRESS_1:PORT'},
{'http': 'http://IP_ADDRESS_2:PORT'},
# Add more proxies as needed
]

# Example of bad email list for filtering
bad_email_list = set([
# Add more known bad emails
])

# Regular expression for basic email syntax validation
email_regex = r'^[a-zA-Z0-9_.+-]+@[a-zA-Z0-9-]+\.[a-zA-Z0-9-.]+$'

def is_valid_email(email):
""" Validate the email format """
return re.match(email_regex, email) is not None

def is_in_bad_list(email):
""" Check if the email is in the bad email list """
return email in bad_email_list

def check_mx_records(domain):
""" Check if the domain has valid MX records """
try:
mx_records = dns.resolver.resolve(domain, 'MX')
return True if mx_records else False
except (dns.resolver.NoAnswer, dns.resolver.NXDOMAIN):
return False

def get_random_proxy():
""" Get a random proxy from the list """
return random.choice(proxies)

def verify_email(email):
""" Perform the full email verification process """
if not is_valid_email(email):
return f"{email} - Invalid Email Format"

if is_in_bad_list(email):
return f"{email} - Found in Bad List"

domain = email.split('@')[1]
if not check_mx_records(domain):
return f"{email} - No MX Records Found"

# Simulate an additional verification step with proxy rotation (optional)
proxy = get_random_proxy()
# Example of making a request through a proxy (customize as needed)
# response = requests.get(f'https://yourverificationurl.com?email={email}', proxies=proxy)
# if response.status_code != 200:
# return f"{email} - Verification Failed with Proxy"

return f"{email} - Verified Successfully"

def verify_email_threaded(email):
""" Wrapper function for threading """
return verify_email(email)

if __name__ == "__main__":
# Example email list for testing
email_list = [
# Add more emails to the list

]

# Using ThreadPoolExecutor to handle multiple emails simultaneously
with concurrent.futures.ThreadPoolExecutor(max_workers=20) as executor:
results = list(executor.map(verify_email_threaded, email_list))

# Print the results
for result in results:
print(result)

The operation is expensive and expected to cost around $2200 per million emails verification and I am looking for a cheaper solution which would be long lasting and sustainable.

Please acknowledge and help.

2 Answers
0

How did you get to your cost and where are you looking at running your script?

EXPERT

answered 2 years ago

  • Well we have talked to a few companies and they gave a rough estimate of the amount I provided above. I am planning to run the script on AWS (for e.g. AWS Lambda) but not sure if it will work as there is a default timeout of 15 mins per lambda.

0

I understand you're looking for a more cost-effective and sustainable solution to verify email deliverability for a large database of email addresses without actually sending emails. While the Python script you've provided is a good start, there are a few considerations and potential improvements to make the process more efficient and less expensive:

  1. Use Amazon SES (Simple Email Service): Instead of building your own email verification system, you can leverage Amazon SES, which provides email address verification capabilities. SES allows you to verify email addresses without sending actual emails, which can be more cost-effective and reliable.

  2. Batch processing: Instead of verifying emails one by one, you can use batch processing to verify multiple email addresses at once, which can significantly reduce the time and resources required.

  3. Implement caching: Store the results of your verifications in a database or cache. This way, you don't need to re-verify email addresses you've already checked, saving time and resources.

  4. Use AWS Lambda: You can create a serverless function using AWS Lambda to run your email verification script. This can be more cost-effective than running a dedicated server, as you only pay for the compute time you use.

  5. Implement rate limiting: To avoid being blocked by email servers, implement rate limiting in your script. This will help ensure your IP doesn't get blacklisted.

  6. Use Amazon SES's bulk verification feature: If you're migrating from another email-sending solution, SES provides a way to verify multiple email addresses at once using a JSON-formatted input file.

  7. Consider using the VerifyEmailIdentity API: This is the recommended method for verifying individual email addresses with Amazon SES.

Remember, while these methods can help reduce costs and improve efficiency, they still involve some level of expense and may not be 100% accurate. Email verification without sending actual emails always involves some level of uncertainty. It's important to balance the need for accuracy with the cost of verification.

Lastly, ensure you're complying with all relevant privacy laws and regulations when handling and verifying email addresses.
Sources
Verifying email addresses in Amazon SES - Amazon Simple Email Service
verify_email_address - Boto3 1.34.114 documentation
Verify multiple email addresses - Amazon Simple Email Service
Verifying an email address - Amazon Simple Email Service

answered 2 years ago

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.