To use an email server on a VPS, you need to set up a mail transfer agent (MTA) like Postfix, configure DNS records (MX, SPF, DKIM, and DMARC), secure the server with SSL/TLS, and connect it to an email client or webmail for sending and receiving emails. Once configured correctly, a VPS-based email server gives you full control over your email system, better deliverability, and improved privacy compared to shared hosting or free email services.
Now, let’s go deeper and explain everything step by step so you can confidently run your own email server on a VPS.
What Is an Email Server on a VPS?
An email server on a VPS (Virtual Private Server) is a self-managed mail system hosted on a virtual machine. Unlike shared hosting email, a VPS provides dedicated resources, a unique IP address, and full root access. This allows you to customize email settings, control spam filters, and scale as your needs grow.
Businesses, marketers, developers, and agencies often use VPS-based email servers for transactional emails, business communication, or bulk mailing (when compliant with laws).
Why Use a VPS for Email Hosting?
Using a VPS for email offers several advantages:
- Full control over configuration and security
- Better deliverability with proper IP and DNS setup
- No shared IP reputation issues
- Custom email limits and automation support
- Higher privacy and data ownership
If you are sending important business emails or running email-based automation, a VPS is a reliable long-term solution.
Prerequisites Before Setting Up an Email Server
Before you start, create sure you have the following:
- A VPS (Ubuntu 20.04 or 22.04 is recommended)
- A registered domain name
- Root or sudo access to the VPS
- A clean VPS IP address (not blacklisted)
- Basic Linux command-line knowledge
How to Use an Email Server on a VPS? Step-by-Step Guide
Step 1: Choose the Right VPS Provider
Not all VPS providers are email-friendly. Select a provider that allows SMTP traffic on port 25 or supports alternative ports like 587.
Important tips when choosing a VPS:
- Ask if SMTP is blocked by default
- Ensure reverse DNS (PTR record) is supported
- Prefer providers with good IP reputation
Popular email-friendly VPS providers include cloud hosting companies that explicitly allow mail servers.
Step 2: Set Hostname and Update the System
Set a proper hostname that matches your domain:
hostnamectl set-hostname mail.yourdomain.com
Then update the server:
apt update && apt upgrade -y
A correct hostname is critical for email trust and spam filtering.
Step 3: Install the Mail Server Software
The most common email stack includes:
- Postfix – for sending emails (MTA)
- Dovecot – for receiving emails (IMAP/POP3)
- OpenSSL – for encryption
Install Postfix:
apt install postfix -y
During installation:
- Select Internet Site
- Enter your domain name (example: yourdomain.com)
Install Dovecot:
apt install dovecot-core dovecot-imapd -y
Step 4: Configure DNS Records (Very Important)
DNS configuration is the backbone of email deliverability.
MX Record
Points email traffic to your mail server.
yourdomain.com → mail.yourdomain.com
A Record
Maps your mail subdomain to the VPS IP.
mail.yourdomain.com → VPS IP
SPF Record
Authorizes your VPS IP to send emails.
v=spf1 ip4:VPS_IP ~all
DKIM Record
Adds cryptographic signing to emails (configured later).
DMARC Record
Tells receivers how to handle failed emails.
v=DMARC1; p=none; rua=mailto:[email protected]
Without these records, your emails will likely land in spam.
Step 5: Secure the Email Server with SSL/TLS
Security is essential to protect credentials and improve trust.
Install Certbot:
apt install certbot -y
Generate SSL certificate:
certbot certonly --standalone -d mail.yourdomain.com
Configure Postfix and Dovecot to use the generated SSL certificates.
This ensures encrypted email transmission and better inbox placement.
Step 6: Create Email Accounts
Email accounts are usually Linux users or virtual mail users.
Example (system user):
adduser info
This creates [email protected].
Dovecot will handle authentication for IMAP/POP3 access.
Step 7: Set Up DKIM for Email Authentication
Install OpenDKIM:
apt install opendkim opendkim-tools -y
Generate DKIM keys and add the public key to your DNS as a TXT record.
DKIM helps receiving servers verify that your emails are not modified and genuinely sent from your domain.
Step 8: Test Email Sending and Receiving
Test email sending from terminal:
echo "Test Email" | mail -s "Email Test" [email protected]
Check:
- Spam folder
- Email headers (SPF, DKIM, DMARC status)
Use tools like mail tester websites to analyze deliverability.
Step 9: Connect Email Clients or Webmail
You can use your VPS email server with:
- Outlook
- Thunderbird
- Apple Mail
- Webmail like Roundcube
IMAP settings example:
- Server: mail.yourdomain.com
- Port: 993
- Encryption: SSL/TLS
SMTP settings:
- Port: 587
- Authentication: Yes
- Encryption: STARTTLS
Step 10: Maintain and Monitor Your Email Server
Ongoing maintenance is crucial:
- Monitor mail logs (/var/log/mail.log)
- Keep the system updated
- Check IP blacklists regularly
- Avoid sending spam or bulk unsolicited emails
- Limit hourly sending rates
A poorly managed server can quickly lose reputation.
Common Mistakes to Avoid
- Skipping SPF, DKIM, or DMARC
- Using a blacklisted VPS IP
- Sending bulk emails without warming up IP
- Not using SSL/TLS
- Ignoring server logs
Avoiding these mistakes ensures long-term email success.
Who Should Use an Email Server on a VPS?
An email server on a VPS is ideal for:
- Small to medium businesses
- SaaS applications (transactional emails)
- Developers and agencies
- Marketers using compliant email campaigns
- Anyone needing full email control
If you want reliability, customization, and independence, VPS email hosting is the right choice.
Final Thoughts
Learning how to use an email server on a VPS gives you complete ownership of your email infrastructure. While setup requires technical effort, the benefits—control, deliverability, security, and scalability—are worth it. With correct DNS, proper authentication, and responsible sending practices, a VPS-based email server can outperform many shared or third-party solutions.
