Ivan Lagunov's Blog

This is a short "recipe" article explaining how to configure remote ElasticSearch instance to support CORS requests and basic authentication using Apache HTTP Server 2.4. Proxy To start with, we need to configure Apache to proxy requests to the Elasticsearch instance. By default, Elasticsearch is running on the port 9200: ProxyPass /elastic http://localhost:9200/ ProxyPassReverse /elastic http://localhost:9200/ Basic authentication Enabling basic authentication is easy. By default, Apache checks the user credentials against the local file which you can create using the following command: /path/to/htpasswd -c /usr/local/apache/password/.htpasswd_elasticsearch elasticsearchuser Then you'll need to use the following directives to allow only authenticated users to access your content: AuthType Basic AuthName "Elastic Server" AuthUserFile /usr/local/apache/password/.htpasswd_elasticsearch Require valid-user For more complex setups such as LDAP-based...

This is a short "recipe" article explaining how to configure basic authentication for the following setup: Apache Tomcat with some application that need be partially password-protected Apache HTTP Server 2.4 as a proxy CentOS 7 Linux server Although basic authentication can be configured within Tomcat itself, my target is to use Apache for that purpose. In addition, as passing unencrypted credentials over the web is insecure, I'm going to install SSL certificates to enable HTTPS for the part of my application. This setup can be used when a part of an internal application need be secured to make it publicly accessible using a separate firewall/proxy (out of scope of this article), that part will be password-protected and SSL-encrypted. Steps Copy certificates into /etc/ssl/certs/ivanlagunov.com Create symlink: cd /etc/httpd sudo ln -s /etc/ssl/certs/ivanlagunov.com Install Apache mod_ssl sudo yum -y install mod_ssl Create file with user credentials for basi...

Quite a while ago I had users complaining they could not use my application from another secure network zone. It appeared the root cause was in using Yahoo CDN for serving YUI resources while there was no internet access in that specific network zone. Also living behind a proxy, our regular users used to suffer from longer delays from time to time due to proxying. An obvious solution turned out to be using a locally served YUI. For this a combo loader is required if you care about efficiency on Production. Installation To start with, there are several alternative open-source tools that can be used for combo loading: Official PHP loader by Yahoo – is obsolete and is said not to work with any version over 3.3.0. CGI script combo – I cannot say much about it besides that it's 3 years old. Node.js combo handler – is kept updated and is the one that I decided to use. The Node.js combo handler is supplied with rather self-contained README file at github. Nevertheless, I...

Once we had a severe issue with Sedna hanging regularly. It was caused by broken indexes after an upgrade at that moment. The issue caused quite a nightmare and led to a lot of time wasted until we solved it together with Sedna devs. Since that moment it has become very important to be able to look into what is happening inside Sedna at any particular moment. Fortunately, there is a suitable way although it's not documented properly on the Sedna website. All you need is to build Sedna from source with a special flag RelWithDebugInfo . Cmake build modes. Using gdb. Using netstat. Cmake build modes Cmake has several build modes with Release and Debug obviously among them. Another mode that can be of big use is called RelWithDebugInfo . There is a perfect explanation for it on the mailing list : The difference between Debug and RelwithDebInfo is that RelwithDebInfo is quite similar to Release mode. It produces fully optimised code, but also builds the program database, and in...

This post lists a number of useful tips and tricks from my daily Linux experience. Mostly I deal with RHEL but I believe these commands are quite independent on Linux distribution (or can be adapted). Network commands Here are network commands represented. Basic net utils: # Who is listening to port: netstat -lp | grep <port> # Show all connections with numeric addresses and proc IDs: netstat -anp # Listen to port (to check connectivity from another side): netcat -l -p <port> # -or- nc -l -p <port> SSH tunnel: # Tunnel to remote_ip:remote_port via proxy_ip with known login/password # The remote_ip:remote_port is being redirected to localhost:local_port ssh -L local_port:remote_ip:remote_port login@proxy_ip # Real-world example of tunnel to remote Sedna XML DB: ssh -L 5050:134.27.100.67:5050 pxqa1@134.27.100.67 Download via HTTP proxy with wget: # Download resource from internet from behind a proxy: http_proxy=http://host:port ; export http_proxy ; w...

This post is actually based on a kind of an epic fail story. Initially the task was just to rename a collection in Sedna XML DB . The solution is as primitive as using RENAME COLLECTION statement of Sedna Data Definition Language. But I'm probably too enthusiastic about writing Bash scripts in Linux. So I missed out single-statement solution and wrote a bunch of scripts to perform the same task via extracting-loading procedure. Anyway, it can still be quite valuable for more complex tasks like moving a collection between XML DB installations (e.g. from Production to Test environment) or merging collections. So my solution follows below. Extracting a single file It's always wise to modularize the code and divide a task into smaller parts. First, we need a script for extracting a single file. It need be parametrized with a file name and a collection name. Also I address another essential problem here that is the safety of file names. It's not a common problem but we do...

In the part 1 of the article I've used scripts to generate bulk load file with LOAD instructions. But that approach has several drawbacks: existing files are not overwritten; hard to track the progress of long-term operation in case of huge number of files. I've written a better script to solve those issues. Bash script for loading files The following Linux Bash script uploads files one by one using separate LOAD instructions . Also it tries to remove the file first using DROP DOCUMENT instruction . As a result, existing files are overwritten. After each 100 of files being loaded, you get a message with a timestamp. It helps to predict the end time of the operation. #!/bin/bash # This function writes a status message to both stdout and $OUTPUT_FILE function print_status { echo ">>> Loaded $counter files, time: `date`" | tee -a $OUTPUT_FILE } OUTPUT_FILE=load_files.log COLLECTION_NAME=legacyBasicTypes echo "" > $OUTPUT_FILE counter=0...

The problem is to upload plenty of files into Sedna XML DB . How would you do this? If it is a repeated action, it's logical to create an application for this. This is quite easy using Sedna XML:DB Java API . Actually we've already done so but this article addresses another case. There is a problem using Java API that is the performance. Using Java API always brings overhead compared to using embedded terminal utility (I got the performance of 2 seconds per file with the remote Sedna installation). Now I have several thousands of files and I want to upload them fast so let's turn to writing some useful scripts to automate it. Generate bulk load file First we need to generate an xquery file with LOAD instructions that are supported by Sedna terminal utility. Let's do this with another simple script. I had to do this under both Linux and Windows systems so you'll find two scripts below. First comes the Linux shell script: #!/bin/sh OUTPUT_FILE=bulk_load.xque...