Category Archives: Uncategorized

One time huge import from Mongo to Solr in Centos 7

1. Install libcurl
yum install curl-devel

2. Install libxml
yum install libxslt-devel

2.1 Make sure both mongo & mongodb php extensions are installed
pecl install mongo
pecl install mongodb
Add the following to php.ini if not already added :
extension=mongo.so
extension=mongodb.so

3. Install solr php extension
pecl install -n solr

4. Enable solr extension
Add extension=solr.so in php.ini and restart apache
sudo service httpd restart
php -m | grep solr

Make sure there are no errors, if there are then it can be related to the order in which files are loaded. In which case remove `extension=solr.so` from php.ini and create a new file in /etc/php.d, possible 41-solr.ini such that it’s loaded at the end and add `extension=solr.so` in that.

Confirm if that file is being used by php.ini by the following command
php -i | grep “.ini”

6. Download scripts from https://github.com/5missions/mongoSolrImporter
Change the .ini file according to your mongo & solr connection settings.
Execute the following command to start the import from mongo to solr
php mongoSolrImporter.php -c mongoSolrImporter.ini

NOTE – The sample .ini file lists the field names in capitals, make sure to have the field name appropriate to your mongo collection. I have all the field-names in lower case in both mongo collection & solr document.

References : http://stackoverflow.com/a/28090368/566092 https://github.com/james75/SolrMongoImporter
https://bugs.php.net/bug.php?id=59267
https://github.com/5missions/mongoSolrImporter

Import huge mongo database from another server in the background

1. Run the following commands :
screen
mongo
use admin
db.runCommand({copydb:1,fromhost:’83.10.17.18:27017′,fromdb:’fromdbname’,todb:’todbname’})
ctrl + a + d (after pressing a in order to exit the screen)

2. Now exiting the SSH session, will still keep the command running. In order to get back to the screen, use the following command
screen -r

Reference : https://docs.mongodb.com/manual/reference/command/copydb/
http://www.tecmint.com/keep-remote-ssh-sessions-running-after-disconnection/
dsad

 

Setup laravel, mongo and solr website on centos 7 having apache and php56

1. Install git
yum install git

2. Install composer
curl -sS https://getcomposer.org/installer | php
chmod +x composer.phar
mv composer.phar /usr/local/bin/composer
composer -V
References : http://idroot.net/tutorials/how-to-install-php-composer-on-centos-7/

https://github.com/petehouston/laravel-deploy-on-shared-hosting

3. Install gcc
yum install gcc

4. Install mongo (php extension)
yum groupinstall “Development tools” -y
yum install openssl-devel
pecl install mongodb

5. Adding mongodb to php.ini
php -i | grep “Loaded Configuration File”
nano /etc/php.ini
Add the following line
extension=mongodb.so
/usr/sbin/apachectl restart
References : http://php.net/manual/en/mongodb.installation.pecl.php

6.
chmod -R o+w storage
composer install
composer dumpautoload -o
php artisan config:cache
php artisan route:cache
References : https://github.com/petehouston/laravel-deploy-on-shared-hosting

7.
php artisan cache:clear
sudo chmod -R 777 vendor storage
References : http://stackoverflow.com/a/38155938/566092

8. Redirect to /public using .htaccess

sudo nano /etc/httpd/conf/httpd.conf
Find the section <directory /var/www/html> and change AllowOverride None to AllowOverride All

<Directory /var/www/html>
AllowOverride All
</Directory>
Save and exit.

Create .htaccess in the root of your web directory with the following contents :

DirectorySlash Off
RewriteEngine on
RewriteRule ^(.*)$ public/$1

Now restart Apache to put the change into effect:
systemctl restart httpd
References : https://devops.profitbricks.com/tutorials/install-and-configure-mod_rewrite-for-apache-on-centos-7/

9. Creating Solr core
sudo -u solr bin/solr create -c video2
References : http://stackoverflow.com/a/36438016/566092

10.  Enabling gzip on apache
Add the following to .htaccess file

<ifmodule mod_deflate.c>
AddOutputFilterByType DEFLATE text/text text/html text/plain text/xml text/css application/x- javascript application/javascript
</ifmodule>
Reference : http://stackoverflow.com/a/5846765/566092

How to setup centos 7 with apache, php56, mongo & solr

Step 1 of 5. Installing Nano

yum install nano

Step 2 of 5. Installing Apache

sudo yum install httpd
sudo systemctl start httpd.service
sudo systemctl enable httpd.service
Step 3 of 5. Installing PHP56
yum -y update
yum -y install epel-release
wget https://dl.fedoraproject.org/pub/epel/epel-release-latest-7.noarch.rpm
wget https://centos7.iuscommunity.org/ius-release.rpm
rpm -Uvh ius-release*.rpm
yum -y update
yum -y install php56u php56u-opcache php56u-xml php56u-mcrypt php56u-gd php56u-devel php56u-mysql php56u-intl php56u-mbstring php56u-bcmath
Step 4 of 5. Install MongoDB
nano vim /etc/yum.repos.d/mongodb.repo
Add the following content in the file
[mongodb]
name=MongoDB Repository
baseurl=http://downloads-distro.mongodb.org/repo/redhat/os/x86_64/
gpgcheck=0
enabled=1

Then exit and save the file with the command :wqyum -y update
yum -y install mongodb-org
systemctl start mongod
systemctl status mongodnano /etc/environment
add these lines:
LANG=en_US.utf-8
LC_ALL=en_US.utf-8mongostat (re-connect using ssh)
mongo (re-connect using ssh)

Reference : http://www.liquidweb.com/kb/how-to-install-mongodb-on-centos-7/
https://docs.mongodb.com/manual/tutorial/install-mongodb-on-red-hat/
http://stackoverflow.com/a/22430837/566092

Step 5 of 5. Installing ApacheSolrcd /tmp
wget http://www-us.apache.org/dist/lucene/solr/6.2.0/solr-6.2.0.tgz
tar xzf solr-6.0.1.tgz solr-6.0.1/bin/install_solr_service.sh –strip-components=2
sudo ./install_solr_service.sh solr-6.0.1.tgz
If the above command doesn’t work, edit install_solr_service.sh as follows :
Edit install_solr_service.sh, find the following line

adduser –system –shell /bin/bash –group –disabled-password –home “$SOLR_VAR_DIR” “$SOLR_USER”
and replace with
adduser –system –shell /bin/bash –home “$SOLR_VAR_DIR” “$SOLR_USER”
yum install lsof
service solr status

 

 

How to use Solarium (3.6.0) with Laravel 5.2

After hours of research falling into traps referring examples of different versions, finally managed to figure out the correct one.

1. Create `app/config/solr.php`

<?php

return array(
    'endpoint' => array(
        'key' => array(
            'host' => '127.0.0.1',
            'port' => 8983,
            'path' => '/solr/',
            'core' => 'mycore'
        )
    )
);

2. Add the following code to HomeController

<?php namespace App\Http\Controllers;  use Illuminate\Support\Facades\Config;  use Solarium\Core\Client\Client;  use Solarium_Exception;  class HomeController extends Controller {      protected $client;          public function __construct()     {          $configSolr = Config::get('solr');          $this->client = new Client($configSolr);
    }
public function index()
{
// create a ping query
$client = $this->client;
$ping = $client->createPing();
// execute the ping query
$result ='';
try {
$result = $client->ping($ping);
echo 'Ping query successful';
echo '<br?>';
var_dump($result->getData());
} catch (Solarium_Exception $e) {
echo 'Ping query failed';
var_dump($e);
}
return $result;
}
}

Coding is a language and should be treated as such

Just discussed with a team-member and was amazed to hear the negligent reply.

Do we speak/say something without knowing what it means to the second person ?

No, right!. So how can we write a line of code without knowing what it does!? Does it make sense ?

Unfortunately, I have been meeting persons, who add hundreds/thousands of lines of code without knowing what it actually does/do. They just add it, because it is meant to do something, but no idea how exactly.

So, whenever in near future, a similar function is required, they tend to copy the same code without taking into consideration the similarity criteria. Just because something very similar is required, doesn’t mean it’s exactly the same as earlier and hence, both pieces of code can be entirely different.

Unless, we realise what a line means, we can never code according to requirements & fix any error or debug any issue/bug.

Why don’t we actually give coding the proper importance of a language ? In my opinion, coding is the easiest language ever made, so it should be the simplest to speak.

Every IT graduate thinks…

Hi friends,

Today I’m going to write about what an IT Graduate dreams to get and what he actually gets and of course the reason behind it.

As a graduate one thinks, to get a job in a nice company with good salary, with weekends (yes I mean both sat & sun) off. No hectic work schedules and a understanding & loving boss.

But does one really deserve this ?
If you have worked hard during your studies, did great preparation for the data-structures, algorithms, network programming, core concepts of software design & architecture, system analysis & design, then of course he deserves it.
But most of us aren’t like that and in physics way, we are not IDEAL and therefore what we get is also not ideal.
We get a low paying job, because our algorithms, programming & data structures aren’t good; an always shouting boss, alternate threatening of salary cuts & getting fired and almost no holidays.

Although unlike reality physics, a small percentage of us does get the ideal lifestyle. You know what’s the difference ?
In my opinion, the difference is just that they struggled very hard to let it move ideally. They were working day and night, no play lifestyle during the precious time when we were busy enjoying our lifestyle (or rather say destroying our future)

Is this fair ? Is there no other option ? OK, I made a mistake, now is there any possible solution ?

There of course is. But since we’ve already screwed things a bit. We need a bigger & better screwdriver to fix it! (if you know what I mean). We can start from taking things one at a time, building the basic blocks of programming, fixing our data-structures and algorithms. As & when it’ll start reflecting in your programs, then your always angry-boss will turn into a cool understanding boss. In a couple of months you’ll see salary increments. Soon your hectic life-style will have lots of rest spaces.

But the only problem is to realize the mistakes made in the past and struggle hard, to correct them in the present. Don’t give up to your frustrating life style instead fight to make it better and more luxurious.

It is not like that I’ve written something new or unique. Each one of us does understand this, but I just mentioned it.

Take all the help in the world you can get

Hi people, today I’m going to share a conversation that I had with one of a trainee in my company. I gave her a task to be completed in a few hours. She did that in 1.5 days and then she wrote me :

sir i created inline stylesheet……….
and i created whole template again at home……
and i didnt take anyone”s help………

I am very surprised on people who feel so proud and prefer to lose time, but not to take help, no matter what. In my personal opinion, it’s the wrong path. We should try to get all help in the world (of course if it’s allowed) to solve a particular problem rather to waste time struggling ourselves. We can at least get the idea/logic of the solution and can try its implementation.

I replied something similar to her :

I don’t care if you take help!… all I care is job should be completed before time with quality! Instead I would say take all help in the world you can get. One might never know the perfect solution, if he just always try to write his own. The art is not about solving the problem, the art is to provide the perfect solution.! :)

Never have this feeling that I won’t take help and build the second Taj Mahal, I had the same attitude, but my teacher told me, Nitin – there are already lots of pending problems to be solved, if you are having one that is already solved, don’t hesitate to ask for solution. Study the solution and may be one day you might be able to solve something that is still unsolved. Just spending most of the time, in solving problems which are already solved, one is wasting a large-time-part of his/her life. So get the solutions, study & analyze them and move to next problem.

To which she replied :

ok sir. You are absolutely right, I will always keep this in my mind……