Saturday, September 24, 2016

Wrong calls and missed calls of Atlanta

I used to get many wrong SMS to my mobile, intended to be sent to Keith. After communicating with those who send them, I realized my number was previously indeed used by someone called Keith. Seems in the US (or at least in GA), they are recycling the mobile number. That means, when I leave, my number will be given to someone else too. I often got missed calls too. After realizing what happened to Keith, his mates stopped messaging or calling my mobile. But worse is yet to be discussed.

There comes the marketing calls to my land line! First they want to sell me security services and auto-insurance. In many cases, they are just bots calling you. Some are in fact scammers. One claimed that they found that my auto-insurance is faulty (I do not even drive!). Another claimed something to do with the US tax authority. They transmit your call to a human, if you answer them patiently. Those humans are far worse than the bots.

The chat goes like "Hello, this is Cathy, how are you doing today?". If you say something like "Hi" or "Hello", the chat proceeds to the next level - immediately to the marketing/sales pitch of the bot. If you say something that cannot be processed by the bot, it will disconnect. Sad part was, the bots called me more frequently than real connections did. :D

One thing I learned was, it is easy to receive calls from these bots. But when you really want to connect to a hospital billing service, it is a night mere. I did this mistake of giving access to two of my debit cards to Piedmont Urgent Care, by WellStreet.

I initially give them my debit card. But later the insurance company sent me a large bill - after paying just 25% or so of the original bill of the hospital. (I learned that the health care in the USA is very expensive. You better not get sick. If you get sick, you better not visit the doctor if at all possible. On top of that, my insurance is a joke - unlike the Swedish one I had in Europe. I should have got a real working insurance). So I asked them to use my HSA savings card to charge me instead of the debit card I previously authorized them to charge, by visiting them again. They accepted my request. What happened next was funny! They charged my debit card despite the request for change well on time and then they also authorized payment for the same amount from the second card (HSA) too, with a pending payment scheduled!

I learned it hard way. Never give access to multiple cards to any vendor. If it is a single card, it has all these security measures in place, mostly. A vendor cannot charge you twice in a single card. The duplicate transaction will usually be failed by the bank. But when you give them access to multiple cards, they of course can charge both your cards, since you are giving away this protection offered by the bank. 

I called the billing service to sort this out, to make sure they are not charging me twice. First, the bot notified me that I must call them Mon - Fri 9 - 5. Then I called them during that time. It starts with this long useless message, "Hi there, Welcome to the WellStreet Urgent Care Services. As our valued customer, your time is very important to us. Listen carefully as most of our options have changed lately. We will be happy to help you.. bla bla bla.. To continue in English, press 1"

Then eventually, once you followed all the shitty things you were expected to do by that moronic phone bot, you get the message that all their customer representatives are busy and cannot attend your call. The bot indicates you to leave your full name, phone number, and details so that some one would get back to you in 24 hours. No one did. Worse, once I somehow managed to get hold of a human after hours of attempt - she disconnected after I mentioned the situation! Probably a honest mistake - she may have mistakenly dropped something, though I suspect that was the case. Eventually, after frustrating days of calling - which I came to realize that there is no customer service exist - I sent them an email to the billing department and the public relations. They immediately cancelled the duplicate payment with an apology.

Finally, email worked better than the phone. Seems they are hiring uneducated and untrained staff for the customer care and billing hotlines. Sometimes these employees are not much better than the bots. Hence coming to my conclusion, here it is easy to get a call from a useless bot. But when you really need to get something done, it is impossible to get hold of a responsible human.

When I was in Sri Lanka, it was easy to assume that the developed countries handle these issues in a better way. When I was in Portugal, it was equally easy to assume that if I am in an English speaking country, the situation would be better. So here I am, in the USA, a developed and English speaking country. Nothing much changed. Bad customer service is everywhere - whether you are in a developed English speaking nation, or a developing alien-language speaking country.

I should of course give credits to awesome customer services offered by many other organizations here - for example, dealing with the GA Power for electricity was always smooth. They have a working web site, and the most helpful social media team ever who is willing to go beyond their duty to give you assistance. So it is all about the teams finally. Not the country actually. Hope for better.

Friday, September 23, 2016

Winning the customers' trust back

A 100$ bonus from Wells Fargo. Any takers?
It was quite noisy here with the Wells Fargo scandal, many asking the CEO to step down. If you are unaware of the news, basically it is a bank in the US, which decided to cheat their customers and stakeholders by faking accounts and credit cards. They reached their business margins at the loss of their customers. In the current world, banks are already earning a lot for just keeping the assets of the customers. Customers trust them. Wells Fargo broke that trust. It needs time to recover. They need a strategy. CEO should of course go, to have overlooked such a mass-scale scam.

On the other hand, I received the above letter. I have lived here in the US for 6 months now, and this is the first time I received a letter from Wells Fargo. I am not sure where they got my contact. But I will assure you - this is not the time to try to acquire more customers. Specially, I am double-suspicious about their 100$ bonus. Too good to be true - from an apparently bogus bank. Pls, first rebuild the trust of your existing customers before trying to get new ones. I am definitely not going to sign up, even if you allure me with 10,000$ bonus!

Query disjoint data bases in parallel and combine, compose, and return the output using Drill

Instead of using MongoDB as a single or a clustered data store, we may partition the data in independent MongoDB instances that are hosted remotely. Then we may use the UNION operator of Drill to join the results accordingly. 

Why do we need to do this? 
i) Because we may already have the data partitioned in different sources.
ii) Due to the domain knowledge, we may do a better job in partitioning the data.
iii) Even in a dumb partitioning, Drill scales and performs well.
iv) There are some interesting research questions, leveraging locality of data to provide better and faster outputs than a clustered or distributed Mongo deployment.

Be warned that Drill has its limitations in data structures that may hurt the performance - for example, nested complex schema such as multi-dimensional arrays. We previously have discussed a work-around for this.
In this post, we will see the simplest example of achieving this.

1. Define the Mongo Storage Plugins
For each of the Mongo Server, define the storage plugin separately in Drill.

Multiple definition of Mongo Storage Plugin, pointing to various Mongo deployments

For example, above mongo3 is defined as below in http://localhost:8047/storage/mongo3

{
  "type": "mongo",
  "connection": "mongodb://184.72.102.246:27017/",
  "enabled": true
}

2. Now query through the query browser:
Querying from the multiple Mongo Deployments and UNION them to the results.



select last_name as id from mongo.employee.empinfo
union all
select first_name as id from mongo2.employee.empinfo
union all
select first_name as id from mongo3.employee.empinfo



Now you may execute this, and get the results. Depending on the nature of the query and partitioning and scale of the data, you may be able to experience performance benefits due to the data partitioning. How do we actually partition the data in each of the MongoDB deployment, with related items co-located in a single partition is a research question, and probably deserves another post.

Monday, September 19, 2016

Drill Integration to Bindaas

Apache Drill has been integrated to Emory BMI's Bindaas Data Server, as a data source provider. The screencast below shows the basic usage of the Drill provider. Please note that the Drill provider is currently experimental and only available in the maven-restructure and maven-restructure-dev branches.

While maven-restructure remains a stable branch following a major restructuring on Bindaas to enhance its usability by the developers, maven-restructure-dev is a branch that is built on top of maven-restructure-dev. Mostly these branches are synced with minor latter developments may only be available at maven-restructure-dev till the merge.

These latter developments will be merged to the master branch eventually. The released versions of Bindaas can be found here.


If your Drill is configured with JPam for authentication, the user of the operating system also functions as the Drill user, as defined in the configurations of your Drill instance.

As Drill driver is based on the JDBC driver, the Drill JDBC url has a similar form. However, user name and password are optional for Drill Provider. If your Drill instance is not configured with JPam, leave the username and password entries blank when you define the data source in the Data Provider Creation step shown in the above screencast.

An example would be,
jdbc:drill:drillbit=localhost:31010 for a Drill configured stand-alone.

P.S: This screencast was captured using gtk-recordmydesktop. It works well on my Ubuntu-16.04. I highly recommend it for your screencasts.

Sunday, September 18, 2016

Building and Running Emory Bindaas on Ubuntu 16.04 LTS

Bindaas is a data services middleware platform built utilizing the componentized architecture of OSGi. Bindaas binary could be downloaded or built from source. To test Bindaas quickly, download the binary and extract it.

Given below is a detailed transcript of what is executed in the above screencast on checking out, building, running, and developing Bindaas with Git, Maven, and IntelliJ IDEA.

Skip the step 1 below and go directly to step 2 if you do not have access to the source code of Bindaas. If you have access to the source code of Bindaas, follow step 1, followed by step 2.

Step 1: Building Emory Bindaas
Alternatively, you may choose to build Bindaas from source. I would recommend using the maven-restructure branch (or maven-restructure-dev branch, if you would like to be in the bleeding edge, though most branches are periodically synced) for that, as it consists of fixes and enhancements to Maven integration. Hence this branch enables easy integration with multiple IDEs such as IntelliJ IDEA, Eclipse, and all the Java IDEs that have Maven integration, where currently master branch supports only Eclipse IDE.
Create Project in Bindaas





Bindaas has been tested with Oracle Java 1.8.x.

Clone the source code.
$ git clone https://{user-name}@bitbucket.org/BMI/bindaas.git

$ cd bindaas

$ git checkout maven-restructure


$ mvn clean install

$ cd dist


Step 2: Running Emory Bindaas
Change to the bin directory of Bindaas distribution (either that is downloaded as binary, or that is built from the source following step 1),
$ cd bin
$ ./startup.sh

This starts Bindaas using nohup. Logs could be found at log/bindaas.log. You may stop the program later by running the shutdown.sh script.

If you rather like to have the logs in the foreground in the terminal itself, instead of using the start up script, you may run the java command directly.

$  java -Dpid=BINDAAS_INSTANCE -Xmx1024m -jar org.eclipse.osgi_3.8.2.v20130124-134944.jar

Now, you may connect to the dashboard at http://localhost:8080/dashboard/ using the user name and password, "admin", "password".

Make sure to have the trailing "/" at the end of URL as shown above.

Once you have created a project using the "Create Project" command, you may define the data providers for the project at http://localhost:8080/dashboard/workspace/{project_name}

You may choose a database such as a mysql or Mongodb databases, or use an HTTP provider.

Once the data provider is defined, queries could be created from the respective listing page of the data provider, 
http://localhost:8080/dashboard/{project_name}/{data_provider_name}/createQueryEndpoint.action

The query could be a simple sql query such as,
select * from states 
"Try Me" on a simple mysql data provider.

Created queries could be viewed from http://localhost:8080/dashboard/{project_name}/{data_provider_name}/query/{query_name} 

"Try Me" option could be used to view the outputs of the query.

This is a very simple post on starting to use Bindaas. You may read further on creating a new data service using Bindaas here, or go through the wiki pages.


Developing Bindaas
First you need to download the entire source tree and build using Maven as in step 1. You may develop using any Java IDE that has Maven integration, including IntelliJ IDEA and Eclipse. I highly recommend IntelliJ IDEA. Just open the pom.xml of the root directory of bindaas source cloned from git to open the entire source hierarchy.

Happy coding!