Skip to main content

Linux command line tips and tricks

This post lists a number of useful tips and tricks from my daily Linux experience. Mostly I deal with RHEL but I believe these commands are quite independent on Linux distribution (or can be adapted).

Network commands
Here are network commands represented.

Basic net utils:
# Who is listening to port:
netstat -lp | grep <port>

# Show all connections with numeric addresses and proc IDs:
netstat -anp

# Listen to port (to check connectivity from another side):
netcat -l -p <port>
# -or-
nc -l -p <port>
SSH tunnel:
# Tunnel to remote_ip:remote_port via proxy_ip with known login/password
# The remote_ip:remote_port is being redirected to localhost:local_port
ssh -L local_port:remote_ip:remote_port login@proxy_ip

# Real-world example of tunnel to remote Sedna XML DB:
ssh -L 5050:134.27.100.67:5050 pxqa1@134.27.100.67
Download via HTTP proxy with wget:
# Download resource from internet from behind a proxy:
http_proxy=http://host:port ; export http_proxy ; wget --proxy=on http://mirror.centos.org/centos/5/os/x86_64/CentOS/mc-4.6.1a-35.el5.x86_64.rpm

# The same for ftp resources:
ftp_proxy=http://host:port ; export ftp_proxy ; wget --proxy=on ftp://ftp.redhat.com/pub/redhat/linux/enterprise/5Server/en/os/SRPMS/mc-4.6.1a-35.el5.src.rpm
Telnet via HTTP proxy:
# 1. Connect to the proxy:
pxqa1@server:/home/pxqa1>telnet myproxy.com 8080
Trying 134.27.0.0...
Connected to myproxy.com (134.27.0.0).
Escape character is '^]'.

## 2. Establish SFTP tunnel
CONNECT remote_sftp_server:22 HTTP/1.0
Proxy-Authorization: Basic bnhw...OQ==

HTTP/1.1 200 Connection established

SSH-2.0-OpenSSH_5.1p1 FreeBSD-20080901

## -or- 2. Establish HTTP tunnel
GET http://www.google.com HTTP/1.0

## -or- 2. Establish FTP tunnel
CONNECT remote_ftp_server:21 HTTP/1.0
Proxy-Authorization: Basic bnhw...OQ==

HTTP/1.1 200 Connection established

220 Test FTP server (version 6.1.1) ready.
USER username
331 Password required for username.
PASS password
230-Welcome to Test!
230 User username logged in.
Transfer data with rsync:
# Copy/update data locally:
rsync -acv --delete source_path destination_path

# Copy/update data remotely via SSH, e.g. war build:
rsync -acv --delete --rsh=ssh .war jboss@134.27.0.0:/usr/local/jboss/server/deploy

# Upload files interruption-safe:
rsync -av --partial --rsh=ssh local_file_name username@remote_host:remote_path
IPTables:
# Show all rules:
iptables -L
# Show all rules with numeric addresses:
iptables -L -n

# Manage service:
service iptables start/stop/status
# Save changes to config file:
service iptables save

# Redirect port, e.g. from 80 to 8080:
iptables -t nat -A PREROUTING -p tcp -m tcp --dport 80 -j REDIRECT --to-ports 8080

# Block particular IP address:
iptables -I INPUT -s 25.55.55.55 -j DROP
# Unblock particular IP address:
iptables -D INPUT -s 25.55.55.55 -j DROP

# Delete chains/rules in table 'nat'
iptables -t nat -F
iptables -t nat -X

Local commands
Here are local commands represented.

Specific finds:
# Find by name with wildcards, e.g. '*.txt'
find . -name "*.txt"

# Find files that contain specific substring, e.g. 'qqq'
find . -exec grep 'qqq' '{}' \; -print

# Find broken symlinks
find . -xdev -type l -print0 | xargs -0 -I '{}' sh -c "[ -e '{}' ] || (echo '{}' is broken)"
One-liners with bash logic:
# Using for loop, e.g. removing all .svn directories recursively
for i in `find -name .svn`; do rm -fr $i ; done

# Using if condition
if [ $t -eq 10 ] ; then echo 'yes'; elif echo 'no'; fi
AWK and SED transformations:
# Just a simple example - extracting time value from ping response
PINGRESPONSE="64 bytes from 172.28.65.253: icmp_seq=1 ttl=128 time=0.221 ms"
TIME=`echo $PINGRESPONSE | awk '{print $7}' | sed 's/time=//'`

Comments

Popular posts from this blog

Connection to Amazon Neptune endpoint from EKS during development

This small article will describe how to connect to Amazon Neptune database endpoint from your PC during development. Amazon Neptune is a fully managed graph database service from Amazon. Due to security reasons direct connections to Neptune are not allowed, so it's impossible to attach a public IP address or load balancer to that service. Instead access is restricted to the same VPC where Neptune is set up, so applications should be deployed in the same VPC to be able to access the database. That's a great idea for Production however it makes it very difficult to develop, debug and test applications locally. The instructions below will help you to create a tunnel towards Neptune endpoint considering you use Amazon EKS - a managed Kubernetes service from Amazon. As a side note, if you don't use EKS, the same idea of creating a tunnel can be implemented using a Bastion server . In Kubernetes we'll create a dedicated proxying pod. Prerequisites. Setting up a tunnel.

Extracting XML comments with XQuery

I've just discovered that it's possible to process comment nodes using XQuery. Ideally it should not be the case if you take part in designing your data formats, then you should simply store valuable data in plain xml. But I have to deal with OntoML data source that uses a bit peculiar format while export to XML, i.e. some data fields are stored inside XML comments. So here is an example how to solve this problem. XML example This is an example stub of one real xml with irrelevant data omitted. There are several thousands of xmls like this stored in Sedna XML DB collection. Finally, I need to extract the list of pairs for the complete collection: identifier (i.e. SOT1209 ) and saved timestamp (i.e. 2012-12-12 23:58:13.118 GMT ). <?xml version="1.0" standalone="yes"?> <!--EXPORT_PROGRAM:=eptos-iso29002-10-Export-V10--> <!--File saved on: 2012-12-12 23:58:13.118 GMT--> <!--XML Schema used: V099--> <cat:catalogue xmlns:cat=

DynamicReports and Spring MVC integration

This is a tutorial on how to exploit DynamicReports reporting library in an existing  Spring MVC based web application. It's a continuation to the previous post where DynamicReports has been chosen as the most appropriate solution to implement an export feature in a web application (for my specific use case). The complete code won't be provided here but only the essential code snippets together with usage remarks. Also I've widely used this tutorial that describes a similar problem for an alternative reporting library. So let's turn to the implementation description and start with a short plan of this how-to: Adding project dependencies. Implementing the Controller part of the MVC pattern. Modifying the View part of the MVC pattern. Modifying web.xml. Adding project dependencies I used to apply Maven Project Builder throughout my Java applications, thus the dependencies will be provided in the Maven format. Maven project pom.xml file: net.sourcefo