Author Archives: admin

One time huge import from Mongo to Solr in Centos 7

1. Install libcurl
yum install curl-devel

2. Install libxml
yum install libxslt-devel

2.1 Make sure both mongo & mongodb php extensions are installed
pecl install mongo
pecl install mongodb
Add the following to php.ini if not already added :
extension=mongo.so
extension=mongodb.so

3. Install solr php extension
pecl install -n solr

4. Enable solr extension
Add extension=solr.so in php.ini and restart apache
sudo service httpd restart
php -m | grep solr

Make sure there are no errors, if there are then it can be related to the order in which files are loaded. In which case remove `extension=solr.so` from php.ini and create a new file in /etc/php.d, possible 41-solr.ini such that it’s loaded at the end and add `extension=solr.so` in that.

Confirm if that file is being used by php.ini by the following command
php -i | grep “.ini”

6. Download scripts from https://github.com/5missions/mongoSolrImporter
Change the .ini file according to your mongo & solr connection settings.
Execute the following command to start the import from mongo to solr
php mongoSolrImporter.php -c mongoSolrImporter.ini

NOTE – The sample .ini file lists the field names in capitals, make sure to have the field name appropriate to your mongo collection. I have all the field-names in lower case in both mongo collection & solr document.

References : http://stackoverflow.com/a/28090368/566092 https://github.com/james75/SolrMongoImporter
https://bugs.php.net/bug.php?id=59267
https://github.com/5missions/mongoSolrImporter

Import huge mongo database from another server in the background

1. Run the following commands :
screen
mongo
use admin
db.runCommand({copydb:1,fromhost:’83.10.17.18:27017′,fromdb:’fromdbname’,todb:’todbname’})
ctrl + a + d (after pressing a in order to exit the screen)

2. Now exiting the SSH session, will still keep the command running. In order to get back to the screen, use the following command
screen -r

Reference : https://docs.mongodb.com/manual/reference/command/copydb/
http://www.tecmint.com/keep-remote-ssh-sessions-running-after-disconnection/
dsad

 

Setup laravel, mongo and solr website on centos 7 having apache and php56

1. Install git
yum install git

2. Install composer
curl -sS https://getcomposer.org/installer | php
chmod +x composer.phar
mv composer.phar /usr/local/bin/composer
composer -V
References : http://idroot.net/tutorials/how-to-install-php-composer-on-centos-7/

https://github.com/petehouston/laravel-deploy-on-shared-hosting

3. Install gcc
yum install gcc

4. Install mongo (php extension)
yum groupinstall “Development tools” -y
yum install openssl-devel
pecl install mongodb

5. Adding mongodb to php.ini
php -i | grep “Loaded Configuration File”
nano /etc/php.ini
Add the following line
extension=mongodb.so
/usr/sbin/apachectl restart
References : http://php.net/manual/en/mongodb.installation.pecl.php

6.
chmod -R o+w storage
composer install
composer dumpautoload -o
php artisan config:cache
php artisan route:cache
References : https://github.com/petehouston/laravel-deploy-on-shared-hosting

7.
php artisan cache:clear
sudo chmod -R 777 vendor storage
References : http://stackoverflow.com/a/38155938/566092

8. Redirect to /public using .htaccess

sudo nano /etc/httpd/conf/httpd.conf
Find the section <directory /var/www/html> and change AllowOverride None to AllowOverride All

<Directory /var/www/html>
AllowOverride All
</Directory>
Save and exit.

Create .htaccess in the root of your web directory with the following contents :

DirectorySlash Off
RewriteEngine on
RewriteRule ^(.*)$ public/$1

Now restart Apache to put the change into effect:
systemctl restart httpd
References : https://devops.profitbricks.com/tutorials/install-and-configure-mod_rewrite-for-apache-on-centos-7/

9. Creating Solr core
sudo -u solr bin/solr create -c video2
References : http://stackoverflow.com/a/36438016/566092

10.  Enabling gzip on apache
Add the following to .htaccess file

<ifmodule mod_deflate.c>
AddOutputFilterByType DEFLATE text/text text/html text/plain text/xml text/css application/x- javascript application/javascript
</ifmodule>
Reference : http://stackoverflow.com/a/5846765/566092

How to setup centos 7 with apache, php56, mongo & solr

Step 1 of 5. Installing Nano

yum install nano

Step 2 of 5. Installing Apache

sudo yum install httpd
sudo systemctl start httpd.service
sudo systemctl enable httpd.service
Step 3 of 5. Installing PHP56
yum -y update
yum -y install epel-release
wget https://dl.fedoraproject.org/pub/epel/epel-release-latest-7.noarch.rpm
wget https://centos7.iuscommunity.org/ius-release.rpm
rpm -Uvh ius-release*.rpm
yum -y update
yum -y install php56u php56u-opcache php56u-xml php56u-mcrypt php56u-gd php56u-devel php56u-mysql php56u-intl php56u-mbstring php56u-bcmath
Step 4 of 5. Install MongoDB
nano vim /etc/yum.repos.d/mongodb.repo
Add the following content in the file
[mongodb]
name=MongoDB Repository
baseurl=http://downloads-distro.mongodb.org/repo/redhat/os/x86_64/
gpgcheck=0
enabled=1

Then exit and save the file with the command :wqyum -y update
yum -y install mongodb-org
systemctl start mongod
systemctl status mongodnano /etc/environment
add these lines:
LANG=en_US.utf-8
LC_ALL=en_US.utf-8mongostat (re-connect using ssh)
mongo (re-connect using ssh)

Reference : http://www.liquidweb.com/kb/how-to-install-mongodb-on-centos-7/
https://docs.mongodb.com/manual/tutorial/install-mongodb-on-red-hat/
http://stackoverflow.com/a/22430837/566092

Step 5 of 5. Installing ApacheSolrcd /tmp
wget http://www-us.apache.org/dist/lucene/solr/6.2.0/solr-6.2.0.tgz
tar xzf solr-6.0.1.tgz solr-6.0.1/bin/install_solr_service.sh –strip-components=2
sudo ./install_solr_service.sh solr-6.0.1.tgz
If the above command doesn’t work, edit install_solr_service.sh as follows :
Edit install_solr_service.sh, find the following line

adduser –system –shell /bin/bash –group –disabled-password –home “$SOLR_VAR_DIR” “$SOLR_USER”
and replace with
adduser –system –shell /bin/bash –home “$SOLR_VAR_DIR” “$SOLR_USER”
yum install lsof
service solr status

 

 

How to use Solarium (3.6.0) with Laravel 5.2

After hours of research falling into traps referring examples of different versions, finally managed to figure out the correct one.

1. Create `app/config/solr.php`

<?php

return array(
    'endpoint' => array(
        'key' => array(
            'host' => '127.0.0.1',
            'port' => 8983,
            'path' => '/solr/',
            'core' => 'mycore'
        )
    )
);

2. Add the following code to HomeController

<?php namespace App\Http\Controllers;  use Illuminate\Support\Facades\Config;  use Solarium\Core\Client\Client;  use Solarium_Exception;  class HomeController extends Controller {      protected $client;          public function __construct()     {          $configSolr = Config::get('solr');          $this->client = new Client($configSolr);
    }
public function index()
{
// create a ping query
$client = $this->client;
$ping = $client->createPing();
// execute the ping query
$result ='';
try {
$result = $client->ping($ping);
echo 'Ping query successful';
echo '<br?>';
var_dump($result->getData());
} catch (Solarium_Exception $e) {
echo 'Ping query failed';
var_dump($e);
}
return $result;
}
}

Coding is a language and should be treated as such

Just discussed with a team-member and was amazed to hear the negligent reply.

Do we speak/say something without knowing what it means to the second person ?

No, right!. So how can we write a line of code without knowing what it does!? Does it make sense ?

Unfortunately, I have been meeting persons, who add hundreds/thousands of lines of code without knowing what it actually does/do. They just add it, because it is meant to do something, but no idea how exactly.

So, whenever in near future, a similar function is required, they tend to copy the same code without taking into consideration the similarity criteria. Just because something very similar is required, doesn’t mean it’s exactly the same as earlier and hence, both pieces of code can be entirely different.

Unless, we realise what a line means, we can never code according to requirements & fix any error or debug any issue/bug.

Why don’t we actually give coding the proper importance of a language ? In my opinion, coding is the easiest language ever made, so it should be the simplest to speak.

Every IT graduate thinks…

Hi friends,

Today I’m going to write about what an IT Graduate dreams to get and what he actually gets and of course the reason behind it.

As a graduate one thinks, to get a job in a nice company with good salary, with weekends (yes I mean both sat & sun) off. No hectic work schedules and a understanding & loving boss.

But does one really deserve this ?
If you have worked hard during your studies, did great preparation for the data-structures, algorithms, network programming, core concepts of software design & architecture, system analysis & design, then of course he deserves it.
But most of us aren’t like that and in physics way, we are not IDEAL and therefore what we get is also not ideal.
We get a low paying job, because our algorithms, programming & data structures aren’t good; an always shouting boss, alternate threatening of salary cuts & getting fired and almost no holidays.

Although unlike reality physics, a small percentage of us does get the ideal lifestyle. You know what’s the difference ?
In my opinion, the difference is just that they struggled very hard to let it move ideally. They were working day and night, no play lifestyle during the precious time when we were busy enjoying our lifestyle (or rather say destroying our future)

Is this fair ? Is there no other option ? OK, I made a mistake, now is there any possible solution ?

There of course is. But since we’ve already screwed things a bit. We need a bigger & better screwdriver to fix it! (if you know what I mean). We can start from taking things one at a time, building the basic blocks of programming, fixing our data-structures and algorithms. As & when it’ll start reflecting in your programs, then your always angry-boss will turn into a cool understanding boss. In a couple of months you’ll see salary increments. Soon your hectic life-style will have lots of rest spaces.

But the only problem is to realize the mistakes made in the past and struggle hard, to correct them in the present. Don’t give up to your frustrating life style instead fight to make it better and more luxurious.

It is not like that I’ve written something new or unique. Each one of us does understand this, but I just mentioned it.

Take all the help in the world you can get

Hi people, today I’m going to share a conversation that I had with one of a trainee in my company. I gave her a task to be completed in a few hours. She did that in 1.5 days and then she wrote me :

sir i created inline stylesheet……….
and i created whole template again at home……
and i didnt take anyone”s help………

I am very surprised on people who feel so proud and prefer to lose time, but not to take help, no matter what. In my personal opinion, it’s the wrong path. We should try to get all help in the world (of course if it’s allowed) to solve a particular problem rather to waste time struggling ourselves. We can at least get the idea/logic of the solution and can try its implementation.

I replied something similar to her :

I don’t care if you take help!… all I care is job should be completed before time with quality! Instead I would say take all help in the world you can get. One might never know the perfect solution, if he just always try to write his own. The art is not about solving the problem, the art is to provide the perfect solution.! :)

Never have this feeling that I won’t take help and build the second Taj Mahal, I had the same attitude, but my teacher told me, Nitin – there are already lots of pending problems to be solved, if you are having one that is already solved, don’t hesitate to ask for solution. Study the solution and may be one day you might be able to solve something that is still unsolved. Just spending most of the time, in solving problems which are already solved, one is wasting a large-time-part of his/her life. So get the solutions, study & analyze them and move to next problem.

To which she replied :

ok sir. You are absolutely right, I will always keep this in my mind……

He who chooses the best doctor but the worse coder

I’m writing my first post related to what flows in my mind.

Do you have clients who always opt to choose the cheapest coder ? Do you have clients who want the best in the least ? Do you have clients who give more priority to the budget than to the quality ?

Well unfortunately, I have quite some.

I have this very strange feeling that why does someone when falls sick wants to get treated by the best doctor in town regardless of his fees. Doesn’t care or think even for a single second for paying 10 times more than an average doctor just to get the best treatment with the most experienced doctor.

Where does this thinking goes when you have to choose someone for writing a software ? When it comes to software, they doesn’t care about the ultimate end-product, all that matters is money. A doctor with 20 years of experience has 10 times fees of a new one. But a coder with 10 years of experience, can’t even quote twice the price of a new bie has quoted. Just because someone thinks that programming knowledge doesn’t matter. Everyone knows how to code. Both the experienced and the newbie will get the job done, so why hire someone experienced at twice the price. Why don’t they think the same for a doctor. Both the new & experienced doctor will get you cured, but of course there’s this probability which says the experienced will give the perfect cure in the least time. Although the newbie might give a cure, but it may not be perfect, it may last for sometime. Soft wares are the same! and hence they suffer from the same disease when one picks up a newbie instead of an experienced coder. Some clients are lucky and they get the software done by the newbie in the cheapest possible price.

But have you had clients who come to you and said – my old developer has gone, he’s not responding, the software he built is no longer functional. I tried this function which I never checked before and it crashes. I want this to be fixed. Now the clients don’t make mistakes, they get the best doctor, since the fever has turned worse into malaria, so only the best doctor can get it cured.

So ultimately they end up paying three times on which they had really saved the time & money, if they had given the job in first hand to the experienced person.

You won’t believe I had the same client, two days ago he told me – “Nitin, they are doing it for me for way less than what you have quoted” and I said – “I’m glad you found someone”. Today, he came and told me that they are stuck somewhere, can you please help. Also regarding the price please talk directly to them about your share in the project budget.

Those coders have already quoted less, my quotation to fix the problem takes almost 80% of their income and now they are pissed, everyone’s leaving the project one-by-one and I’m sure that the client shall come back to me straight away and ask to start it again from scratch.

This is a note for newbies – Don’t put a quote for something that you can’t do. Otherwise you may end up regretting your decision.

Note for the clients – It’s perfectly fine to pay more to experienced coders just as similar to an experienced doctor.

When I started writing I thought it’ll take hardly 2-3 lines, but woh! it’s almost 500 words now. Anyways, good coders don’t fear if you don’t get the job. The client shall ultimately land upon you sooner or later. Newbies, don’t be in a hurry to earn a fortunate, things take time, give time the time and you’ll be an experienced coder one day ;)