Linux Blog » Linux tips

ReiserFS for scalable and high load SMTP servers

Linux — Tue, 19 Aug 2008 02:07:03 +0000

We have seen great performance on using ReiserFS for mail system spools and in our tests ReiserFS file system spool for Postfix queues was very fast. For best performance we used

notail, noatime mount options in /etc/fstab file.

Full example:

/dev/sda2/usr reiserfs notail,noatime,rw,auto 0 0

notatime - turns off atimes calls, reduces IO and greatly increases file system performance
notail - increases performance of ReiserFS especially for small files - works great for mail servers, web servers and caches. This option disables file pckaging in tree.

Iptable ip_conntrack table set-up and tunning for high load UDP traffic

Linux — Sat, 22 Sep 2007 21:24:10 +0000

If you run a busy DNS server or any other service that uses a lot of UDP traffic, it’s possible that your default Iptable conntrack sessions (connection tracking entries in kernel memory) settings are too low and netfilter is unable to track all your sessions.

The error is usually something like this:

Sep 20 11:53:44 hostname01 kernel: ip_conntrack: table full, dropping packet.

You need to tune sysctl net.ipv4.ip_conntrack_max value, let’s say increase it twice or more times and see if you still get the error messages on the console or syslog.

Depending on your OS, the formula for calculating the maximum number of conntrack sessions your box can handle is as follows:

The size of each session record really depends on the kernel config and many other compile options. For 2.6.* kernels it is around 300 bytes.

You can also easily check out the current usage of connection tracking

wc -l /proc/net/ip_conntrack

Default sysctl settings for Red Hat Enterprise boxes, possible the same applies for Fedora and Centos.

net.ipv4.netfilter.ip_conntrack_tcp_max_retrans = 3
net.ipv4.netfilter.ip_conntrack_tcp_be_liberal = 0
net.ipv4.netfilter.ip_conntrack_tcp_loose = 3
net.ipv4.netfilter.ip_conntrack_tcp_timeout_max_retrans = 300
net.ipv4.netfilter.ip_conntrack_log_invalid = 0
net.ipv4.netfilter.ip_conntrack_generic_timeout = 600
net.ipv4.netfilter.ip_conntrack_icmp_timeout = 30
net.ipv4.netfilter.ip_conntrack_udp_timeout_stream = 180
net.ipv4.netfilter.ip_conntrack_udp_timeout = 30
net.ipv4.netfilter.ip_conntrack_tcp_timeout_close = 10
net.ipv4.netfilter.ip_conntrack_tcp_timeout_time_wait = 120
net.ipv4.netfilter.ip_conntrack_tcp_timeout_last_ack = 30
net.ipv4.netfilter.ip_conntrack_tcp_timeout_close_wait = 60
net.ipv4.netfilter.ip_conntrack_tcp_timeout_fin_wait = 120
net.ipv4.netfilter.ip_conntrack_tcp_timeout_established = 432000
net.ipv4.netfilter.ip_conntrack_tcp_timeout_syn_recv = 60
net.ipv4.netfilter.ip_conntrack_tcp_timeout_syn_sent = 120
net.ipv4.netfilter.ip_conntrack_checksum = 1
net.ipv4.netfilter.ip_conntrack_buckets = 8192
net.ipv4.netfilter.ip_conntrack_count = 18988
net.ipv4.netfilter.ip_conntrack_max = 34576

You can decrease the net.ipv4.netfilter.ip_conntrack_tcp_timeout_established, by half, at least.

sysctl -w net.ipv4.netfilter.ip_conntrack_tcp_timeout_established=216000

I hope this helps you with your Linux server network stack tunning. Good luck!

Improving Linux network stack

Linux — Tue, 22 May 2007 13:31:27 +0000

The settings for sysctl.conf below apply for Fedora, RedHat, Centos OS as well as other Linux flavors. These settings will improve your server network stack and some little protection against ddos attacks as well. Feel free to improve as you wish :)

# Kernel sysctl configuration file for Red Hat Linux
#
# For binary values, 0 is disabled, 1 is enabled. See sysctl(8) and
# sysctl.conf(5) for more details.

# Disables packet forwarding
net.ipv4.ip_forward=0

# Disables IP source routing
net.ipv4.conf.all.accept_source_route = 0
net.ipv4.conf.lo.accept_source_route = 0
net.ipv4.conf.eth0.accept_source_route = 0
net.ipv4.conf.default.accept_source_route = 0

# Enable IP spoofing protection, turn on source route verification
net.ipv4.conf.all.rp_filter = 1
net.ipv4.conf.lo.rp_filter = 1
net.ipv4.conf.eth0.rp_filter = 1
net.ipv4.conf.default.rp_filter = 1

# Disable ICMP Redirect Acceptance
net.ipv4.conf.all.accept_redirects = 0
net.ipv4.conf.lo.accept_redirects = 0
net.ipv4.conf.eth0.accept_redirects = 0
net.ipv4.conf.default.accept_redirects = 0

# Enable Log Spoofed Packets, Source Routed Packets, Redirect Packets
net.ipv4.conf.all.log_martians = 0
net.ipv4.conf.lo.log_martians = 0
net.ipv4.conf.eth0.log_martians = 0

# Enable IP spoofing protection, turn on source route verification
net.ipv4.conf.all.rp_filter = 1
net.ipv4.conf.lo.rp_filter = 1
net.ipv4.conf.eth0.rp_filter = 1
net.ipv4.conf.default.rp_filter = 1

# Disable ICMP Redirect Acceptance
net.ipv4.conf.all.accept_redirects = 0
net.ipv4.conf.lo.accept_redirects = 0
net.ipv4.conf.eth0.accept_redirects = 0
net.ipv4.conf.default.accept_redirects = 0

# Disables the magic-sysrq key
kernel.sysrq = 0

# Decrease the time default value for tcp_fin_timeout connection
net.ipv4.tcp_fin_timeout = 15

# Decrease the time default value for tcp_keepalive_time connection
net.ipv4.tcp_keepalive_time = 400

# Turn off the tcp_window_scaling
net.ipv4.tcp_window_scaling = 0

# Turn off the tcp_sack
net.ipv4.tcp_sack = 0

# Turn off the tcp_timestamps
net.ipv4.tcp_timestamps = 0

# Enable TCP SYN Cookie Protection
net.ipv4.tcp_syncookies = 1

# Lower syn retry rates
net.ipv4.tcp_synack_retries = 2
net.ipv4.tcp_syn_retries = 3

# Enable ignoring broadcasts request
net.ipv4.icmp_echo_ignore_broadcasts = 1

# Enable bad error message Protection
net.ipv4.icmp_ignore_bogus_error_responses = 1

# Log Spoofed Packets, Source Routed Packets, Redirect Packets
net.ipv4.conf.all.log_martians = 1

# Increases the size of the socket queue (effectively, q0).
net.ipv4.tcp_max_syn_backlog = 1024

# Increase the tcp-time-wait buckets pool size
net.ipv4.tcp_max_tw_buckets = 1440000

# Allowed local port range
net.ipv4.ip_local_port_range = 16384 65536