Wednesday, December 26, 2018

JAVA - Calculate distance between two wells' coordinates for Pad calculation

This is my first attempt to write JAVA code. could be better and better :)
This code loop through each one of the API in one file to 
get the distance between each one of them.


package com.nan.lens.wellcost;

import java.io.IOException;
import java.io.InputStream;
import java.io.InputStreamReader;
import java.util.*;

import org.apache.commons.csv.CSVFormat;
import org.apache.commons.csv.CSVRecord;

public class Pad {

    private APIProvider provider;

    public Pad(APIProvider apiProvider) {
        this.provider = apiProvider;
    }

    public static Pad create() {
        try {
            return new Pad(new APIProvider("/apiFile.csv"));
        } catch (IOException e) {
            /*The throw statement creates a new object*/            
            throw new RuntimeException(e);
        }
    }

    // read data into java from csv    
     public static class APIProvider {
        // create map to store API_coordinates pair        
        private HashMap<String, ArrayList<Double>> API_LAT_LONG_1 = new HashMap<>();
        private HashMap<String, ArrayList<Double>> API_LAT_LONG_2;
        // create list to store coordinates        
        private ArrayList<Double> LAT_LONG = new ArrayList<Double>();
        private ArrayList<Double> temp = new ArrayList<Double>();

        public APIProvider(String apiFile) throws IOException {

            InputStream input = Pad.class.getResourceAsStream(apiFile);
            Iterable<CSVRecord> records = CSVFormat.EXCEL.parse(new InputStreamReader(input));

            for (CSVRecord record : records) {
                String key = record.get(0);
                temp.clear();
                /*Force to change type String to Double*/                
                temp.add(0, Double.valueOf(record.get(1)));
                temp.add(1, Double.valueOf(record.get(2)));
                LAT_LONG = (ArrayList<Double>) temp.clone();
                API_LAT_LONG_1.put(key, LAT_LONG);
            }
            //System.out.println("API_LAT_LONG: " + API_LAT_LONG_1);            
            API_LAT_LONG_2 = (HashMap) API_LAT_LONG_1.clone();

            HashMap<String, ArrayList<String>> API_DIST = new HashMap<>();
            ArrayList<String> api_dist_list;
            ArrayList<String> temp_list = new ArrayList<String>();
            final int R = 6371; // Radius of the earth in km
            for (String key : API_LAT_LONG_1.keySet()) {
                //get lat and long from ArrayList                
                List<Double> list = new ArrayList<Double>();
                list = API_LAT_LONG_1.get(key);
                Double lat1 = list.get(0);
                Double lng1 = list.get(1);

                for (String key2 : API_LAT_LONG_2.keySet()) {
                    List<Double> list2 = new ArrayList<Double>();
                    list2 = API_LAT_LONG_2.get(key2);
                    Double lat2 = list2.get(0);
                    Double lng2 = list2.get(1);

                    // Calculate distance between two points                    
                    double latDistance = Math.toRadians(lat2 - lat1);
                    double lonDistance = Math.toRadians(lng2 - lng1);
                    double a = Math.sin(latDistance / 2) * Math.sin(latDistance / 2)
                            + Math.cos(Math.toRadians(lat1)) * Math.cos(Math.toRadians(lat2))
                            * Math.sin(lonDistance / 2) * Math.sin(lonDistance / 2);
                    double c = 2 * Math.atan2(Math.sqrt(a), Math.sqrt(1 - a));
                    double distance = R * c; //km
                    distance = Math.pow(distance, 2);

                    if (key != key2 && distance < 0.0762) { // (km)                        
                        String UID = UUID.randomUUID().toString();
                        temp_list.clear();

                        temp_list.add(0, key);
                        temp_list.add(1, key2);
                        temp_list.add(2, String.valueOf(distance));

                        api_dist_list = (ArrayList<String>) temp_list.clone();
                        API_DIST.put(UID, api_dist_list);
                    }
                }
            }
            System.out.println(API_DIST);

        }
    }
}

Tuesday, December 4, 2018

Apache Kafka for beginners

https://www.cloudkarafka.com/blog/2016-11-30-part1-kafka-for-beginners-what-is-apache-kafka.html#


Apache Kafka and server concepts

Here are important concepts that you need to remember before we dig deeper into Apache Kafka - explained in one line.

  • Producer: Application that sends the messages.
  • Consumer: Application that receives the messages.
  • Message: Information that is sent from the producer to a consumer through Apache Kafka.
  • Connection: A connection is a TCP connection between your application and the Kafka broker.
  • Topic: A Topic is a category/feed name to which messages are stored and published.
  • Topic partition: Kafka topics are divided into a number of partitions, which allows you to split data across multiple brokers.
  • Replicas A replica of a partition is a "backup" of a partition. Replicas never read or write data. They are used to prevent data loss.
  • Consumer Group: A consumer group includes the set of consumer processes that are subscribing to a specific topic.
  • Offset: The offset is a unique identifier of a record within a partition. It denotes the position of the consumer in the partition.
  • Node: A node is a single computer in the Apache Kafka cluster.
  • Cluster: A cluster is a group of nodes i.e., a group of computers.