Unpacking the Containers versus Virtual Machine Debate

Posted on by admin

Virtual machines virtualize the hardware and containers virtualize the operating system kernels.

    Containers:

    • Do not emulate any hardware and do not need to boot up.
    • Do not require operating system installation.
    • Take up much less space.
    • Can run only one app at a time (by design).
    • Can interact with their hosts.

      Virtual Machines:

      • Uses the hypervisor to emulate real hardware.
      • Can take up a lot of space.
      • Require you to install/configure operating system.
      • Can run multiple apps at the same time.
      • Cannot interact with their hosts.

      Pros of Docker Containers:

      • Portability: The containers can run on any system which has Docker installed which makes it easy to move the application from one environment to another.
      • Lightweight: The containers take less resources in comparison to virtual machines, which makes it easier to deploy applications in resource-constrained environments.
      • Efficiency: A single host can manage multiple containers while launching and stopping containers significantly quicker than virtual machines.
      • Isolation: It is feasible to run many containers on the same host without worrying about application conflicts since containers provide an isolation level that is suitable for many applications.

      Cons of Docker Containers:

      • Security: Containers are vulnerable to security risks that target the host operating system since they rely on the host operating system for security.
      • Persistence: Data must be stored in an external storage solution or a separate volume because containers do not by default persist data.
      • Complexity: Especially when it comes to network and storage management, managing a large number of containers may be difficult and complex.

      Pros of Virtual Machines:

      • Isolation: Virtual machines provide a high level of isolation between the host and guest operating systems, enabling multiple virtual machines to run on the same host without risk of application incompatibilities.
      • Compatibility: It is feasible to run legacy apps that may not be compatible with the host operating system because virtual machines can run any operating system.
      • Security: Applications and data security are made simpler by the ability of virtual machines to be segregated from the host operating system and other virtual machines.
      • Persistence: Virtual machines persist data by default, making it easier to manage data for long-term storage.

        Cons of Virtual Machines:

        • Resource Utilization: Virtual machines consume more resources compared to containers, making them less efficient for resource-constrained environments.
        • Management Overhead: Virtual machines require more management overhead compared to containers, making it necessary to install and configure guest operating systems, virtual network interfaces, and virtual storage devices.
        • Performance Overhead: Virtual machines introduce a performance overhead compared to running applications directly on the host operating system, making it necessary to balance performance and isolation requirements.

        Power Query: Revolutionizing the ETL Process

        Posted on by admin

        About Power Query

        Power Query is a data analysis tool developed and owned by Microsoft. It is part of the Microsoft Power BI suite, a collection of business intelligence and data visualization tools. Power Query was originally developed as an add-on for Microsoft Excel, but has since been integrated into the Power BI suite and is now also available in other Microsoft products, such as Power Pivot and Power View.

        What is Power Query

        Extracting, transforming, and loading (ETL) data is a crucial step in the data analysis process. The quality and accuracy of the insights you gain from your data is directly dependent on the quality and accuracy of the data itself. The traditional method of ETL, which involves manual data entry, is time-consuming, prone to errors, and often requires a high level of technical expertise. However, Power Query has revolutionized the ETL process, making it faster, more efficient, and less prone to errors.

        The ETL process can also be automated with Power Query. Users can save and reuse their queries, eliminating the need to manually perform the same data extraction, transformation, and loading steps every time they need to analyze a new data set. This not only saves time but also helps to reduce the risk of human error.

        Benefits

        One of the most significant benefits of Power Query is its ability to handle large data sets with ease. It can quickly extract data from different sources, such as Excel spreadsheets, databases, web pages, and more, and then transform and clean that data to meet the specific needs of your analysis.

        Another significant advantage of Power Query is its ability to automate the ETL process. Users can save and reuse their queries, which eliminates the need to manually perform the same data extraction, transformation, and loading steps every time they need to analyze a new data set.

        Functions

        Power Query also provides a range of powerful data transformation functions, such as pivot, group, merge, and aggregate. These functions allow users to manipulate and shape data in meaningful ways to fit their analysis requirements. Additionally, Power Query supports a range of data visualization options, including pivot tables, charts, and graphs, which makes it easy to create interactive reports and dashboards.

        One of the standout features of Power Query is its ability to handle data from a wide range of sources. It supports a wide range of file formats, such as CSV, Excel, and XML, and also provides a range of data connectors for popular databases, such as SQL Server, Oracle, and MySQL. Additionally, Power Query allows users to access data from online services, such as Google Analytics, Salesforce, and SharePoint, which makes it easy to integrate data from different systems.

        In conclusion, Power Query has revolutionized the ETL process by making it faster, more efficient, and less prone to errors. Its simple and intuitive user interface, powerful data transformation functions, and support for a wide range of data sources make it a must-have tool for anyone who needs to perform data analysis. Whether you’re a data analyst, data scientist, or business intelligence professional, Power Query is the key to unlocking insights from your data.

        AI Tool: ChatGPT is leading the way in AI

        Posted on by admin

        What is ChatGPT?

        Simply a language model trained by OpenAI. It is an artificial intelligence model designed to assist with answering questions on a wide range of topics. GPT stands for Generative Pre-trained Transformer Language Model. The tool generates natural language text responses based on users’ text prompts. The model was trained on a large dataset of human conversations.

        ChatGPT uses a transformer architecture and the model is trained using unsupervised learning, which means it is able to learn about language patterns and structure from the data itself, without being explicitly told what to do. This allows it to generate human-friendly responses that are appropriate for a given context.

        Several use cases for this model, enhance the overall customer experience and streamline business operations. It can be fine-tuned for specific domains such as customer service, technical support, or general conversations. ChatGPT provides much better and in-depth information in-comparison to Google search results and also you can get more refined information from the search results.

        One of the major drawbacks is that the model is trained on data until 2021 and it is not connected to the internet or any live data streams. Therefore, it cannot answer questions about current events after 2021.

        To Get Started…

        • Browse to the website chat.openai.com.
        • OpenAI will prompt to sign up or sign in if you already got an account.
        • ChatGPT is free now, but in future it might not be free. Check your usage page from your profile.
        • Once completed with registration or login, OpenAI will take you to the main ChatGPT page.
        • Write your questions on the textbox and you will get prompts.
        • Have fun playing with it.
        • Warning: Do not type your personal data or valuable information.

        Prompts

        Format: Write a [length] [writing result] on [topic] in the style of [style]

        Examples:

        • Write a poem to cheer my mood in this rainy and stormy weather
        • Rephrase this text: [type your text]
        • Provide me with an outline of the book [name of the book]
        • Replace my simple vocabulary with advanced English [type you text]
        • Act as a professional spelling and grammar corrector and improver
        • Write a customer email
        • Write a resume/cover letter

        Understanding FinTech

        Posted on by admin

        FinTech – Financial technology is the intersection of technology to deliver financial services such as software to manage financial accounts, make payments or transfer money using smartphones and using analytics to make investment decisions.

        The Evolution of FinTech

        • FinTech 1.0 : Infrastructure
        • FinTech 2.0 : Banks
        • FinTech 3.0 : Startups
        • FinTech 3.5 : Emerging Markets

        FinTech 2.0: Banks (1967)

        • Evolution of finance and technology
        • First automated teller machine (ATM)
        • First handheld calculator
        • Process of digitization
        • Establishment of a series of domestic and international electronic payment systems in the late 1960s and early 1970s
        • Establishment of NASDAQ the world’s first electronic stock exchange in 1971
        • Online banking system in 1980
        • Program Trading in 1987

        FinTech 3.0: Startups and FinTech 3.5: Emerging Markets

        • Global financial crisis – 2008 (real estate market crash)
        • Blockchain – crypto and bitcoin
        • Distributed ledger technologies
        • Peer-to-peer landing
        • Crowdfunding
        • Mobile payment with Smartphones

        How 2008 Impacted the development of FinTech?

        • Unemployment
        • Regulatory changes
        • Distrust in Bank
        • Invention of Smartphones – launched in 2007

        FinTech Typology

        • Finance and Investment

        Alternative to Financing: Crowdfunding, P2P Lending, ICOs (Initial Coin Offerings)

        • Operations and Risk Management

        Pre 2008 financial crisis the focus was on Quantitative Risk Modelling Techniques eg. VAR (Value-at- Risk Systems) and after 2008 the financial services spend heavily on IT.

        • Payments and Infrastructure

        Electronic payments

        • Data Security and Monetization

        Datafication such as Big Data or Artificial Intelligence and Risk such as Hacking and Cyber Security

        • Technology in the Customer Interface

        New business technologies

        Four Phases of Money

        Barter System

        In the 9000BC to 600BC with the limitations of the capacity of carrying goods, transporting it across long distance to make exchanges and standardize commence.

        Commodity Money

        In 1100BC, China used small bronze replicas of goods as token, while some countries also used silver and gold. In Rome, soldiers were paid in salt as their salary.

        Coinage

        Coinage was the system of creating and issuing coins as a form of currency. The coinage system was introduced by Lydians, currently known as Turkey around 600BC. China introduced paper money in 700AD and Europe did likewise in the year 16000AD

        Dematerialized Payments

        In the 1950s as the technology advanced and became more widely available, digital payments became more common and the volume of non-cash transaction has been steadily increasing ever since then.

        Payment Evolution

        • 1946 – Credit Cards
        • 1950 – Cheques
        • 1980 – ATM
        • 1990 – EMV Standards
        • 2000 – Mobile money, e-wallets and virtual cards
        • 2009 – Bitcoin
        • 2010 – Contactless Cards
        • 2015 – Wearable devices, mobile wallets, other cryptocurrencies

        Real-Time Gross Settlement (RTGS)

        Real-time gross settlement (RTGS) systems enable the immediate exchange of large amounts of money between banks. These systems, which are often run by central banks or other financial institutions, are used for high-value transactions. Large sums of money may be transferred quickly and reliably between financial institutions because to RTGS systems’ rapid, secure, and effective architecture. RTGS systems are used to settle a variety of transactions, such as large-value business payments, government securities settlements, and interbank transfers. This is only possible because of technological capacities and customer expectation of instant delivery.

        February 2022 Market Overview

        Posted on by admin
        Nasdaq S&P 500 Dow
        Feb 1 14346.00(+106.12) 4546.54(+30.99) 35405.24(+273.38)
        Feb 3 13878.81(-538.73) 4477.44(-111.94) 35111.16(-518.17)

        Key Highlights

        Feb 3:

        • Strong: Consumer Staples
        • Weak: Communication Services, Consumer Discretionary, Information Technology
        • Meta’s disappointing earning weakens the tech sector

        Feb 1:

        • Strong: Energy, Materials, Financials, Industrials
        • Weak: Utilities, Real Estate, Consumer Staples
        • Strong earnings from XOM, Google and UPS.

        Power Apps

        Posted on by admin

        What is Power Apps?

        • Is a tool of Microsoft Power Platform to develop applications.
        • You can use Power Apps with Power Automate and Power BI to build powerful applications.
        • Collection of apps, services, connectors.
        • A data platform to build customs apps according to your business needs.
        • Custom apps can connect to data platform or online and on-premises data sources such as SharePoint , Excel, Office 365, Dynamics, SQL Server.
        • Allows users to build apps without writing code.
        • Benefit of using power app is that it allows many data sources in a single app.

        Power Apps Components

        Power Apps Home Page

        • Here you can develop apps from sample apps, templates or blank screen.

        Power Apps Studio

        • Here you can fully customise your app to meet your business goals.

        Power Apps Admin Center

        • Plaform to manage your power apps. Define your environments, data connections, manage environment role and data policies.

        Data sources, connections, and gateways

        • Popular data sources: common data service, sharepoint, sql server, dynamic 365, office 365 users, office 365 outlook, excel, one drive for business, onedrive, dropbox.
        • Some connectons allow power apps to read and write data.
        • Some cloud services such as salesforce or twitter can be a datasource.
        • For on-premises data, you use gateway to provide reliable connection. The gateway will be implemented on a non-premises computer and communicate with power apps.

        January 2022 Market Overview

        Posted on by admin

        NasdaqS&P 500Dow
        Jan – 315832.79(+187.83)4796.56(+30.38)36585.06(+246.76)
        Jan – 415622.71(-210.08)4793.54(-3.02)36799.65(+214.59)
        Jan – 5 15100.17(-522.54)4700.58(-92.96)36407.11(-392.54)
        Jan – 6 15080.86(-19.31)4696.05(-4.53)36236.47(-170.64)
        Jan – 7 14935.9(-144.96)4677.03(-19.02)36231.66(-4.81)
        Week 2
        Jan – 1014942.83(+6.93)4670.29(-6.74)36068.87(-162.79)
        Jan – 1115153.45(+210.62)4713.07(+42.78)36252.02(+183.15)
        Jan – 12 15188.39(+34.94) 4726.35(+13.28) 36290.32(+38.30)
        Jan – 1314806.81(-381.58) 4659.03(-67.32) 36113.62(-176.70)
        Jan – 1414893.75(+86.94) 4662.85(+3.82) 35911.81(-201.81)
        Week 3
        Jan 1814506.89(-386.86) 4577.11(-85.74) 35368.47(-543.34)
        Jan 1914340.25(-166.64) 4532.76(-44.35) 35028.65(-339.82)
        Jan 2014154.02(-186.23) 4482.73(-50.03) 34715.39(-313.26)
        Jan 2113768.92(-385.10) 4397.94(-84.79) 34265.37(-450.02)
        Week 4
        Jan 2413855.13(+86.21) 4410.13(+12.19) 34364.50(+99.13)
        Jan 2513539.30(-315.83)4356.45(-53.68) 34297.73(-66.77)
        Jan 2613542.12(+2.82) 4349.93(-6.52) 34168.09(-129.64)
        Jan 2713352.78(-189.34) 4326.51(-23.42) 34160.78(-7.31)
        Jan 2813770.57(+417.79) 4431.85(+105.34) 34725.47(+564.69)
        Week 5
        Jan 31 14239.88(+469.31) 4515.55(+83.70) 35131.86(+406.39)

        Movers

        Jan 31

        • Strong: Consumer Discretionary, Information Technology, Communication Services
        • Weak: Energy, Consumer Staples
        • Tech stocks rebound

        Jan 28

        • Strong: Information Technology, Communication Services, Real Estate
        • Weak: Energy
        • Investor concern: inflation and interest rates
        • End of month balance sheet adjustments

        Jan 27

        • Strong: Consumer Staples, Energy, Utilities, Materials
        • Weak: Consumer Discretionary, Real Estate, Information Technology, Financials
        • Fed expectations saga continued today to close the market in red
        • Semiconductor sector is weak
        • Supply chain issues causing tech sector to decline
        • Airline industry challenges high wages, fuel cost, maintenance cost and airport parking

        Jan 26

        • Strong: Information Technology, Financials
        • Weak: Real Estate, Materials, Communication Services, Industrials
        • FOMC press conference – roll back policy support and risk of high inflation in coming months
        • FOMC rates near zero and will end asset purchase in march 20200

        Jan 25

        • Strong: Energy, Financials
        • Weak: Information Technology, Consumer Discretionary, Communication Services, Utilities
        • Investor waiting for FOMC policy decision tomorrow

        Jan 24

        • Strong: Consumer Discretionary, Energy, Industrials
        • Weak: Utilities, Health Care, Consumer Staples
        • Omicron scare, FED interest rate fear, Mixed corporate results
        • Bounce from oversold result
        • Ukraine war fear

        Jan 21

        • Stock market in red
        • Strong: Consumer Staples
        • Weak: Communication Services, Consumer Discretionary, Materials
        • Fed and geopolitics

        Jan 20

        • Weak: Consumer Discretionary, Materials, Information Technology, Industrials
        • Selling pressure

        Jan 19

        • Strong: Consumer Staples, Utilities
        • Weak: Financials, Consumer Discretionary, Information Technology
        • Selling pressure from all the sectors
        • Increase in weekly MBA Mortgage Application Index by 2.3%
        • Increase in Housing starts by 1.4% m/m
        • Increase in permits by 9.1% m/m

        Jan 18

        • Strong: Energy
        • Weak: Information Technology, Financials, Communication Services
        • Inflation pressure and interest rates hike.
        • 2-yr yield rise 1.00%, 10-yr yield flirts with 1.88%
        • Empire State Manufacturing Survey for January dropped to -0.7 from 31.9 in December.
        • NAHB Housing Market Index for January decreased to 83 from 84 in December.

        Jan – 14

        • Strong: Energy, Information Technology, Communication Services
        • Weak: Real Estate, Financials, Materials, Utilities
        • Strength in the mega-caps stocks
        • Mixed bank earnings, downbeat economic data, interest rates rise

        Jan – 13

        • Strong: Industrials, Utilities, Consumer Staples
        • Weak: Information Technology, Consumer Discretionary, Health Care, Communication Services
        • Growth stocks selling interest.
        • Airlines outperform after beating estimates.
        • PPI increased by 0.2% m/m.
        • Initial claims for the week ending January 8 increased by 23,000 to 230,000.

        Jan – 12

        • An increase in CPI data 0.5% month-over-month in December and 7.0% year-over-year. Sharpest 12-month increase since June 1982. Core CPI, which excludes food and energy, rose 0.6% month-over-month and was up 5.5% year-over-year. This was the sharpest 12-month increase since February 1991.
        • The Treasury Budget showed a $21.3 bln deficit in December versus a $143.6 bln deficit in the same period a year ago. December marked the 27th consecutive month that the Treasury has seen a budget deficit.
        • The budget deficit over the last 12 months is $2.58 trln versus a deficit of $2.70 trln in November.
        • The weekly MBA Mortgage Applications Index increased 1.4% following a 5.6% decline in the prior week.
        • Weekly EIA crude oil inventories decreased by 4.55 mln barrels after decreasing by 2.14 mln barrels during the previous week.
        • Fed’s Beige Book for January reported that economic activity in the U.S. expanded at a modest pace in the last weeks of 2021 and that some districts observed a deceleration in the robust price increases from the previous months.
        • Jan-11 : Fed Chair Powell’s Senate confirmation hearing. Fed will end asset purchases in March, hike rates over the course of the year and allow the balance sheet to run off later in the year.
        • Jan-5 : Fed Minutes from the December meeting showed that participants thought it would be appropriate to reduce the size of the Fed’s balance sheet at a faster pace than during the previous normalization period.
        • Jan-4: Daily COVID-19 cases in the U.S. topped one million, ISM Manufacturing Index decelerated more than expected to 58.7% from 61.1% in November, it still denoted an expanding manufacturing sector.
        • Jan- 3: Higher oil prices , Big tech earnings, tax loss selling pressure.

        TSQL – Using Pivot Table

        Posted on by admin

        A Pivot table in TSQL is useful to spread row data as columns. Row values can be defined into multiple columns.

        A row in SQL Server represents an entity and a column as an attribute of the entity.

        To build a pivot table, you will need to define three columns.

        • Column 1 – Row grouping
        • Column 2 – Create new columns based on the values
        • Column 3 – Aggregated values for the row and columns

        Steps to build a pivot table:

        1. Select based dataset
        2. Create temporary result set using Common Expression Table (CTE)
        3. Apply PIVOT operator

        Example

        This is the original table

        Rate_IdDescR01R02R03R04
        1Plan14.1025.2028.5040.50
        2Plan21.5039.5041.8060.90

        This will be the final result

        PlanSilverGold
        Plan-114.1021.50
        Plan-225.2039.50
        Plan-328.5041.80
        Plan-440.5060.90

        Step 1: Create the table

        CREATE TABLE #Rate (
Rate_Id int,
Rate_Desc varchar(100),
R01 decimal(17,2),
R02 decimal(17,2),
R03 decimal(17,2),
R04 decimal(17,2)
);

        Step 2: Insert records

        INSERT INTO #Rate VALUES (1,'Plan',14.10,25.20,28.50,40.50);
INSERT INTO #Rate VALUES (2,'Plan',21.50,39.50,41.80,60.90);

        Step 3: Use cross apply to create additional rows

        SELECT * FROM #Rate
cross apply
(
   select 1,R01 union
   select 2,R02 union
   select 3,R03 union
   select 4,R04 
) r (RowNo, Rate)

        Output

        Rate_IdRate_DescR01R02R03R04RowNoRate
        1Plan14.1025.2028.5040.50114.10
        1Plan14.1025.2028.5040.50225.20
        1Plan14.1025.2028.5040.50328.50
        1Plan14.1025.2028.5040.50440.50
        2Plan21.5039.5041.8060.90121.50
        2Plan21.5039.5041.8060.90239.50
        2Plan21.5039.5041.8060.90341.80
        2Plan21.5039.5041.8060.90460.90

        Step 4: Use PIVOT to change rows to columns

        WITH CTE_Rates As (
SELECT * FROM #Rate
cross apply
(
   select 1,R01 union
   select 2,R02 union
   select 3,R03 union
   select 4,R04 
) r (RowNo, Rate))
SELECT [Plan], [1] as [Silver], [2] as [Gold]
FROM (
SELECT Rate_Id,CONCAT(Rate_Desc,'-',RowNo) as [Plan], Rate_Desc, RowNo,Rate
FROM CTE_Rates) r
PIVOT(
 Max(Rate)
  for Rate_Id IN ([1],[2])
) p
;

        Final table result from Step 4

        PlanSilverGold
        Plan-114.1021.50
        Plan-225.2039.50
        Plan-328.5041.80
        Plan-440.5060.90

        In this example I have converted Rate_Id 1 and 2 as Silver and Gold. Also R01, R02, R03, R04 as Plan-1, Plan-2, Plan-3, Plan-4.

        Code #1: Reverse An Array

        Posted on by admin

        How to reverse an array (in C#) without using Array.Reverse() method?

        int[] arr = {1,3,4,9,8};
for (int i = 0; i < arr.Length / 2; i++)
{
int tmp = arr[i];
arr[i] = arr[arr.Length - i - 1];
arr[arr.Length - i - 1] = tmp;
}
Console.WriteLine(string.Join(",", arr));

        Market Movers – July 2020

        Posted on by admin

        July 6

        • ISM Non-Manufacturing Index for June increased to 57.1 (Note: above 50 is expansion)

        July 2

        • June non-farm payroll jobs increased by 4.8 million.
        • A decline in the unemployment rate to 11.1%.
        • Sector performance: Energy (2.95%), Financials (2.17%), Industrials (1.98%), Materials (1.97%), Consumer Discretionary (1.33%), Information Technology (1.31%), Consumer Staples (1.20%), Real Estate (1.04%), Health Care (0.85%), Comminucation Services (0.72%), Utilities (0.59%)

        July 1

        • Promising signs of COVID vaccine by Pfizer Link
        • ISM Manufacturing Index 52.6 (above 50 is Expansion) Link
        • Private sector Payroll Gain (2.37 million jobs) Link
        • FOMC’s June meeting Link

        Credit Cards: