Programming Application & Framework

Version Control System

Version control systems are a category of software tools that help a software team manage changes to source code over time.

Version control software keeps track of every modification to the code in a special kind of database.

If a mistake is made, developers can turn back the clock and compare earlier versions of the code to help fix the mistake while minimizing disruption to all team members.

For almost all software projects, the source code is like the crown jewels - a precious asset whose value must be protected.

For most software teams, the source code is a repository of the invaluable knowledge and understanding about the problem domain that the developers have collected and refined through careful effort.

Version control protects source code from both catastrophe and the casual degradation of human error and unintended consequences.

Software developers working in teams are continually writing new source code and changing existing source code.

The code for a project, app or software component is typically organized in a folder structure or "file tree".

One developer on the team may be working on a new feature while another developer fixes an unrelated bug by changing code, each developer may make their changes in several parts of the file tree.

Version control helps teams solve these kinds of problems, tracking every individual change by each contributor and helping prevent concurrent work from conflicting.

Changes made in one part of the software can be incompatible with those made by another developer working at the same time.

This problem should be discovered and solved in an orderly manner without blocking the work of the rest of the team. Further, in all software development, any change can introduce new bugs on its own and new software can't be trusted until it's tested.

So testing and development proceed together until a new version is ready.

Good version control software supports a developer's preferred workflow without imposing one particular way of working.

Ideally, it also works on any platform, rather than dictate what operating system or toolchain developers must use.

Great version control systems facilitate a smooth and continuous flow of changes to the code rather than the frustrating and clumsy mechanism of file locking - giving the green light to one developer at the expense of blocking the progress of others.

Software teams that do not use any form of version control often run into problems like not knowing which changes that have been made are available to users or the creation of incompatible changes between two unrelated pieces of work that must then be painstakingly untangled and reworked.

If you're a developer who has never used version control you may have added versions to your files, perhaps with suffixes like "final" or "latest" and then had to later deal with a new final version.

Perhaps you've commented out code blocks because you want to disable certain functionality without deleting the code, fearing that there may be a use for it later.

Version control is a way out of these problems.

Version control software is an essential part of the every-day of the modern software team's professional practices.

Individual software developers who are accustomed to working with a capable version control system in their teams typically recognize the incredible value version control also gives them even on small solo projects.

Once accustomed to the powerful benefits of version control systems, many developers wouldn't consider working without it even for non-software projects.

Benefits Of Version Control System

Developing software without using version control is risky, like not having backups.
Version control can also enable developers to move faster and it allows software teams to preserve efficiency and agility as the team scales to include more developers.

Version Control Systems (VCS) have seen great improvements over the past few decades and some are better than others.
VCS have sometimes known as SCM (Source Code Management) tools or RCS (Revision Control System).
One of the most popular VCS tools in use today is called Git. Git is a Distributed VCS, a category known as DVCS, more on that later.
Like many of the most popular VCS systems available today, Git is free and open source. Regardless of what they are called, or which system is used, the primary benefits you should expect from version control are as follows.

A complete long term change history of every file. This means every change made by many individuals over the years. Changes include the creation and deletion of files as well as edits to their contents. Different VCS tools differ on how well they handle renaming and moving of files. This history should also include the author, date and written notes on the purpose of each change . Having the complete history it is crucial when needing to fix problems in older versions of the software. If the software is being actively worked on, almost everything can be considered an "older version " of the software.
Branching and merging. Having team members work concurrently is a no-brainer, but even individuals working on their own can benefit from the ability to work on independent streams of changes. Creating a "branch" in VCS tools keeps multiple streams of work independent from each other while also providing the facility to merge that work back together, enabling developers to verify that the changes on each branch do not conflict. Many software teams adopt a practice of branching for each feature or perhaps branching for each release, or both. There are many different workflows that teams can choose from when they decide how to make use of branching and merging facilities in VCS.
Traceability. Being able to trace each change made to the software and connect it to project management and bug tracking software such as Jira, and being able to annotate each change with a message describing the purpose and intent of the change help not only with the root cause analysis and other forensics. Having the annotated history of the code at your fingertips when you are reading the code, trying to understand what it is doing and why it is so designed can enable developers to make correct and harmonious changes that are in accord with the intended loge term design of the system.this can be especially important for working effectively with legacy code and is crucial in enabling developers to estimate future work with any accuracy.

While it is possible to develop software without using any version control, doing so subject the project to a huge risk that no professional team would be advised to accept. So the question is not whether to use version control but which version control system to use.

There are two types of VCS: centralized and distributed.

Centralized Version Control

A centralized version control system works on a client-server model.

There is a single, (centralized) master copy of the code base, and pieces of the code that are being worked on are typically locked, (or “checked out”) so that only one developer is allowed to work on that part of the code at any one time.

Access to the code base and the locking is controlled by the server. When the developer checks their code back in, the lock is released so it’s available for others to check out.

Of course, an important part of any VCS is the ability to keep track of changes that are made to the code elements, and so when an element is checked in, a new version of that piece is created and logged.

When everyone has finished working on their different pieces and it’s time to make a new release, a new version of the application is created, which usually just means logging the version numbers of all the individual parts that go together to make that version of the application.

Probably the best-known examples of centralized VCS systems are CVS and Subversion, both of which are open source, although there have been many commercial examples (including IBM’s Rational ClearCase).

Teamstudio CIAO! also falls into this category, but I’ll come back to that later.

Distributed Version Control

More recently, there’s been a trend (or some might call it a revolution) toward distributed version control systems.

These systems work on a peer-to-peer model: the code base is distributed amongst the individual developers’ computers.

In fact, the entire history of the code is mirrored on each system.

There is still a master copy of the code base, but it’s kept on a client machine rather than a server.

There is no locking of parts of the code; developers make changes in their local copy and then, once they’re ready to integrate their changes into the master copy, they issue a request to the owner of the master copy to merge their changes into the master copy.

With a DVCS, the emphasis switches from versions to changes, and so a new version of the code is simply a combination of a number of different sets of changes.

That’s quite a fundamental change in the way many developers work, which is why DVCS’s are sometimes considered harder to understand than centralized systems.

The best-known examples of distributed VCS’s are Git and Mercurial, both of which are open source.

Pros and Cons

⭐The key difference between centralized and distributed VCS’s, in my opinion, revolves around the fact that there is no locking of elements in a distributed system.

So every new set of changes that a developer makes is essentially like a new branch of the code, that needs to be merged back into the master repository.

In the distributed model, it’s possible for two developers to be working on the same source file at the same time.

⭐That one fundamental difference means that:

Performance of distributed systems is better because there is no waiting for locks to happen across potentially slow network connections.Also, the complete code base is already on your local system.

Branching and merging are much easier to achieve in a distributed system, largely because it’s built into the way the system works.

With a distributed system, you don’t need to be connected to the network all the time.

⭐For these reasons, many people have become DVCS fans and believe that Git and Mercurial are the answer to any VCS question. However, centralized systems like Subversion do have some benefits too:

Centralized systems are typically easier to understand.

Access control is easier since everything is controlled from one place (the server).

Unless you want to, you don’t have to merge different versions of the same code, which can be tricky.

Git VS GitHub

Git also provides for interaction to a centralized location for collaborative development and project backup.

And for those new to Git, this is where the confusion begins.

GitHub is a wonderful place that allows you to publish your trade secrets to the world.

Okay, that’s how some people may look at it—it’s actually much more than that.

If you are concerned that your code is so unique that it doesn’t already exist somewhere else, GitHub allows you to create private repositories, albeit not many for free.

So what is GitHub, really? Simply put, it is a repository where you can get a free account for storing Git-controlled software.

This does not have to be limited to application development either, as Git allows you to use version control for any electronic media, such as documents.

GitHub provides a convenient place to store multiple versions of these files.

Git is a revision control system, a tool to manage your source code history.

GitHub is a hosting service for Git repositories.

So they are not the same thing: Git is the tool, GitHub is the service for projects that use Git.

It Matters

For students, this is invaluable because it allows instructors and peers to review your code and interact with it. In fact, they can download your code and run it just as you do. It can not be overemphasized how useful this is when you are discussing your code remotely. Students are also able to practice collaboration on large, useful projects with the same method as many development teams. It’s also very useful when pursuing a career since it gives a place to showcase your work to prospective employers.

Git Commands, Commit and Push

Compare and contrast

Basically, git commit "records changes to the repository" while git push "updates remote refs along with associated objects". So the first one is used in connection with your local repository, while the latter one is used to interact with a remote repository.

Here is a nice picture from Oliver Steele, that explains the git model and the commands:

Read more about git push and git pull on GitReady.com (the article I referred to first)

Staging Area

The staging area is a simple file, generally contained in your Git directory, that stores information about what will go into your next commit. It’s sometimes referred to as the index, but it’s becoming standard to refer to it as the staging area.

Use of Staging Area and Git Directory

🌞There are many uses of staging in git. Some are listed below:-

staging helps you split up one large change into multiple commits -🠊 Let's say you worked on a large-ish change, involving a lot of files and quite a few different subtasks.

You didn't actually commit any of these -- you were "in the zone", as they say, and you didn't want to think about splitting up the commits the right way just then. (And you're smart enough not to make the whole thing on honking big commit!).

Now the change is all tested and working, you need to commit all this properly, in several clean commits each focused on one aspect of the code changes.
With the index, just stage each set of changes and commit until no more changes are pending.

Really works well with git gui if you're into that too, or you can use git add -p or, with newer gits, git add -e.

staging helps in reviewing changes -🠊 Staging helps you "check off" individual changes as you review a complex commit, and to concentrate on the stuff that has not yet passed your review.
Let me explain. Before you commit, you'll probably review the whole change by using git diff.
If you stage each change as you review it, you'll find that you can concentrate better on the changes that are not yet staged.
git gui is great here.
It's two left panes show unstaged and staged changes respectively, and you can move files between those two panes (stage/unstage) just by clicking on the icon to the left of the filename.
Even better, you can even stage partial changes to a file.
In the right pane of git gui, right click on a change that you approve of and choose "stage hunk".
Just that change (not the entire file) is now staged; in fact, if there are other, unstaged, changes in that same file, you'll find that the file now appears on both top and bottom left panes!

staging helps when a merge has conflicts - 🠊When a merge happens, changes that merge cleanly are updated both in the staging area as well as in your work tree.

Only changes that did not merge cleanly (i.e., caused a conflict) will show up when you do a git diff, or in the top left pane of git gui.

Again, this lets you concentrate on the stuff that needs your attention -- the merge conflicts.

staging helps you keep extra local files hanging around -🠊 Usually, files that should not be committed go into .gitignore or the local variant, .git/info/exclude.

However, sometimes you want a local change to a file that cannot be excluded (which is not good practice but can happen sometimes).

For example, perhaps you upgraded your build environment and it now requires an extra flag or option for compatibility, but if you commit the change to the Makefile, the other developers will have a problem.

Of course, you have to discuss with your team and work out a more permanent solution, but right now, you need that change in your working tree to do any work at all! Another situation could be that you want a new local file that is temporary, and you don't want to bother with the ignore mechanism. This may be some test data, a log file or trace file, or a temporary shell script to automate some test... whatever. In git, all you have to do is never to stage that file or that change. That's it.

staging helps you sneak in small changes -🠊 Let's say you're in the middle of a somewhat large-ish change and you are told about a very important bug that needs to be fixed asap.

The usual recommendation is to do this on a separate branch, but let's say this fix is really just a line or two and can be tested just as easily without affecting your current work.

With git, you can quickly make and commit only that change, without committing all the other stuff you're still working on. Again, if you use git gui, whatever's on the bottom left pane gets committed, so just make sure only that change gets there and commit, then push!

Workflow

🌠A Git Workflow is a recipe or recommendation for how to use Git to accomplish work in a consistent and productive manner.

Git workflows encourage users to leverage Git effectively and consistently.

Git offers a lot of flexibility in how users manage changes.

Given Git's focus on flexibility, there is no standardized process on how to interact with Git.

When working with a team on a Git managed project, it’s important to make sure the team is all in agreement on how the flow of changes will be applied.

To ensure the team is on the same page, an agreed upon Git workflow should be developed or selected.

There are several publicized Git workflows that may be a good fit for your team. Here, we’ll be discussing some of these workflow options.

The array of possible workflows can make it hard to know where to begin when implementing Git in the workplace.

This page provides a starting point by surveying the most common Git workflows for software teams.

As you read through, remember that these workflows are designed to be guidelines rather than concrete rules.

We want to show you what’s possible, so you can mix and match aspects from different workflows to suit your individual needs.

Successful Git Workflow

🌟When evaluating a workflow for your team, it's most important that you consider your team’s culture.
You want the workflow to enhance the effectiveness of your team and not be a burden that limits productivity.

Some things to consider when evaluating a Git workflow are:

★Does this workflow scale with team size?
★Is it easy to undo mistakes and errors with this workflow?
★Does this workflow impose any new unnecessary cognitive overhead to the team?

Centralized Workflow

The Centralized Workflow is a great Git workflow for teams transitioning from SVN.
Like Subversion, the Centralized Workflow uses a central repository to serve as the single point-of-entry for all changes to the project.
Instead of a trunk, the default development branch is called master and all changes are committed into this branch.
This workflow doesn’t require any other branches besides master.

Transitioning to a distributed version control system may seem like a daunting task, but you don’t have to change your existing workflow to take advantage of Git.
Your team can develop projects in the exact same way as they do with Subversion.

However, using Git to power your development workflow presents a few advantages over SVN.

First, it gives every developer their own local copy of the entire project.

This isolated environment lets each developer work independently of all other changes to a project - they can add commits to their local repository and completely forget about upstream developments until it's convenient for them.

Second, it gives you access to Git’s robust branching and merging model.

Unlike SVN, Git branches are designed to be a fail-safe mechanism for integrating code and sharing changes between repositories. The Centralized Workflow is similar to other workflows in its utilization of a remote server-side hosted repository that developers push and pull form. Compared to other workflows, the Centralized Workflow has no defined pull request or forking patterns. A Centralized Workflow is generally better suited for teams migrating from SVN to Git and smaller size teams.

How it works

Developers start by cloning the central repository.

In their own local copies of the project, they edit files and commit changes as they would with SVN; however, these new commits are stored locally - they’re completely isolated from the central repository.

This lets developers defer synchronizing upstream until they’re at a convenient break point.

To publish changes to the official project, developers "push" their local master branch to the central repository.

This is the equivalent of svn commit, except that it adds all of the local commits that aren’t already in the central master branch.

Initialize the Central Repository

⧬First, someone needs to create the central repository on a server.

If it’s a new project, you can initialize an empty repository.

Otherwise, you’ll need to import an existing Git or SVN repository.

⧬Central repositories should always be bare repositories (they shouldn’t have a working directory), which can be created as follows:

⧪ssh user@host git init --bare /path/to/repo.git

⧪Be sure to use a valid SSH username for user, the domain or IP address of your server for host, and the location where you'd like to store your repo for /path/to/repo.git.

Note that the .git extension is conventionally appended to the repository name to indicate that it’s a bare repository.

Hosted central repositories

Central repositories are often created through 3rd party Git hosting services like Bitbucket Cloud or Bitbucket Server.
The process of initializing a bare repository discussed above is handled for you by the hosting service.
The hosting service will then provide an address for the central repository to access from your local repository.

Clone the central repository

Next, each developer creates a local copy of the entire project.

This is accomplished via the git clone command:

🔼git clone ssh://user@host/path/to/repo.git

When you clone a repository, Git automatically adds a shortcut called origin that points back to the “parent” repository, under the assumption that you'll want to interact with it further on down the road.

Make changes and commit

Once the repository is cloned locally, a developer can make changes using the standard Git commit process:

🔼 edit, stage, and commit.

If you’re not familiar with the staging area, it’s a way to prepare a commit without having to include every change in the working directory.

This lets you create highly focused commits, even if you’ve made a lot of local changes.

git status # View the state of the repo

git add <some-file> # Stage a file

git commit # Commit a file</some-file>

Remember that since these commands create local commits, John can repeat this process as many times as he wants without worrying about what’s going on in the central repository.
This can be very useful for large features that need to be broken down into simpler, more atomic chunks.

Push new commits to the central repository

Once the local repository has new changes committed.
These change will need to be pushed to share with other developers on the project.

git push origin master

This command will push the new committed changes to the central repository.
When pushing changes to the central repository, it is possible that updates from another developer have been previously pushed that contain code which conflict with the intended push updates.
Git will output a message indicating this conflict. In this situation, git pull will first need to be executed.
This conflict scenario will be expanded on in the following section.

Managing conflicts

The central repository represents the official project, so its commit history should be treated as sacred and immutable.

If a developer’s local commits diverge from the central repository, Git will refuse to push their changes because this would overwrite official commits.

Before the developer can publish their feature, they need to fetch the updated central commits and rebase their changes on top of them.

This is like saying, “I want to add my changes to what everyone else has already done.”
The result is a perfectly linear history, just like in traditional SVN workflows.

If local changes directly conflict with upstream commits, Git will pause the rebasing process and give you a chance to manually resolve the conflicts.

The nice thing about Git is that it uses the same git status and git add commands for both generating commits and resolving merge conflicts.

This makes it easy for new developers to manage their own merges. Plus, if they get themselves into trouble, Git makes it very easy to abort the entire rebase and try again (or go find help).

Example

Let’s take a general example at how a typical small team would collaborate using this workflow.
We’ll see how two developers, John and Mary, can work on separate features and share their contributions via a centralized repository.

Content Delivery Networks

Content Delivery Networks (CDN) is a system of servers deployed in different geographical locations to handle increased traffic loads and reduce the time of content delivery for the user from servers.
The main objective of CDN is to deliver content at top speed to users in different geographic locations and this is done by a process of replication.
CDNs provide web content services by duplicating content from other servers and directing it to users from the nearest data center.
The shortest possible route between a user and the web server is determined by the CDN based on factors such as speed, latency, proximity, availability and so on.
CDNs are deployed in data centers to handle challenges with user requests and content routing.

CDNs are used extensively by social networks, media and entertainment websites, e-commerce websites, educational institutions, etc. to serve content quickly to users in different locations.

1. Your reliability and response times get a huge boost

High performing website equals high conversion and growing sales.
Latency and speed issues tend to cripple web businesses and cause damage.
A few seconds can mean the difference between a successful conversion or a bounce.
A reliable CDN ensures that the load speed is more than optimal and that online transactions are made seamlessly.

2. A CDN enables global reach

Over one third of the world’s population is online, which means that the global use of the internet has increased exponentially over the last 15 years.
CDNs provide solutions through cloud acceleration with local POPs.
This global reach will eliminate any latency problems that interrupt long-distance online transactions and cause slow load times.

3. A CDN saves a lot of money

Hiring a CDN results in noticeable savings for a business; rather than investing in an infrastructure and separate service providers all across the globe, a global CDN can eliminate the need to pay for costly foreign hosting and thus, save your business a lot of money.
A global CDN offers a single platform to handle all of the separate operations, working across numerous regions for a reasonable price.
CDNs are also recommended for companies with a tight budget.

4. 100% percent availability

Due to the distribution of assets across many regions, CDNs have automatic server availability sensing mechanisms with instant user redirection.
As a result, CDN websites experience 100 percent availability, even during massive power outages, hardware issues or network problems.

5. Decrease server load

The strategic placement of a CDN can decrease the server load on interconnects, public and private peers and backbones, freeing up the overall capacity and decreasing delivery costs.
Essentially, the content is spread out across several servers, as opposed to offloading them onto one large server.

6. 24/7 customer support

Quality CDNs have been known for outstanding customer support. In other words, there is a CS team standby at all time, at your disposal.
Whenever something occurs, you have backup that’s waiting to help you fix your performance related problems.
Having a support team on quick dial is a smart business decision – you’re not just paying for a cloud service, you’re paying for a large spectre of services that help your business grow on a global scale.

7. Increase in the number of Concurrent Users

Strategically placing the servers in a CDN can result in high network backbone capacity, which equates to a significant increase in the number of users accessing the network at a given time.
For example, where there is a 100 GB/s network backbone with 2 tb/s capacity, only 100 GB/s can be delivered.
However, with a CDN, 10 servers will be available at 10 strategic locations and can then provide a total capacity of 10 x 100 GB/s.

8. DDoS protection

Other than inflicting huge economic losses, DDoS attacks can also have a serious impact on the reputation and image of the victimized company or organization.
Whenever customers type in their credit card numbers to make a purchase online, they are placing their trust in that business.
DDoS attacks are on the rise and new ways of Internet security are being developed; all of which have helped increase the growth of CDNs, as cloud security adds another layer of security.
Cloud solutions are designed to stop an attack before it ever reaches your data center. A CDN will take on the traffic and keep your website up and running.
This means you need not be concerned about DDoS attacks impacting your data center, keeping your business’ website safe and sound.

9. Analytics

Content delivery networks can not only deliver content at a fast pace, they can also offer priceless analytical info to discover trends that could lead to advertising sales and reveal the strengths and the weaknesses of your online business.
CDNs have the ability to deliver real-time load statistics, optimize capacity per customer, display active regions, indicate which assets are popular, and report viewing details to their customers.
These details are extremely important, since usage logs are deactivated once the server source has been added to the CDN.
Info analysis shows everything a developer needs to know to further optimize the website.
In-depth reporting ultimately leads to performance increase, which results in higher user experience and then further reflects on sales and conversion rates.

Hiring a CDN is a growing trend in the internet community.
Performance and security are everything – a CDN is here to deliver that.
A high performance website creates income, growth, web presence and brand awareness.
If your web business is suffering, you should consider hiring a CDN today.

Web Hosting

There are different methods and technologies to build a web hosting server.
These days, we can’t simply talk only about web servers as most of the websites have applications like MySQL, Oracle, MS SQL etc.
So, it is total hosting solution that includes web servers, as well as application servers.
According to your traffic (hits) you can decide which solution to go for.
The basic web hosting solution is shared hosting.
If you have more traffic, and needs more performance you may need to go with VPS (Virtual Private Server), Dedicated Hosting or Could Hosting.
Dedicated hosting solution is expensive, but its an ideal solution for high performance applications and high traffic environment.

Differences Between CDNs and Web Hosting

(1) Web Hosting is to host your web in a server to allow people to access from Internet, whereas CDN increases the delivery speed of your web content across the world.

(2) CDN at the moment deliver only the static part of your website, but Google is planning to cache the whole page including content of your web pages, web servers on the other hand, contain all your web related content.

(3) Mostly, web content are hosted in a single server, but CDN content will be spread across the world, in multiple hosted environment.

Virtualization Goals

• Virtualization goals: (Goldberg(1974) first survey paper on virtualization technologies.)

➪Efficiency: Virtual Machine must be as efficient as the non-virtualized machine, if not more!

➪Isolation: Virtual Machine is an isolated duplicate of the real machine.

➪Safety: Virtual Machine can access resources explicitly allocated to it only

Desirables - Hardware Virtualizer

• Need for hardware virtualizer that supports:

➢System hygiene: Virtual machines should not be built using instruction trap and simulation method since such approaches are clumsy and awkward.

➢Software simplicity: Inherent hardware support for virtualization would make the VMM a small and simple program that further contributes to system reliability and security.

➢System performance: Systems that are designed to support virtualization should operate more efficiently than their non-virtualized counterparts.

f-map virtual resource definition

• The f-map transforms the virtual resource name to its corresponding real resource name.

• f-map is transparent to the VM in its state of execution in privileged or non-privileged mode.
Hence, the VM executes as it would on a real machine.

• All real and virtual resources are under the control of the VMM. VMM manipulates and invokes the f-map, and any f-map violation passes the control to the VMM.

Design Requirements for virtualizability

• System management is achieved using OS kernel functions executing in privileged CPU execution mode.

• To support one level of virtualization three distinct privilege levels of execution are required:
⇰ Hypervisor

• Highest privilege

• Controls and manages all physical resources ⇰ GuestOS

• De-privileged when compared to the hypervisor

• Controls and manages all virtual resources allocated to the specific VM

• Traps to Hypervisor kernel through hypervisor calls(?) for mapping and realizing virtual resource functions through physical resource access ⇰ GuestUser

• Normal user mode privilege

• Traps to GuestOS kernel through system calls for virtual resource access

• All sensitive instructions must be privileged to enable this trap approach.

x86 Architecture suitability for Virtualization

• Early x86 architectures supported only two privilege levels, execution and protected mode.

• Some of the sensitive instructions were not privileged as a result outcome of instruction depended on the privilege mode.

• MMU recognized two levels of address translation
➠User level virtual address to kernel level physical address
➠System TLB also supported just two level addressing

• All I/O instructions were privileged and were
executed only in the protected mode.

• Overall system thus designed for hosting a single OS and system resources were abstracted using OS constructs for access using the process abstraction.

Virtualization Extension to x86 Architecture

• Support for atleast four CPU privilege levels (also called rings)

• Support for extended page tables (2-level)

• MMU hosted TLB that could map Guest User Address to Hypervisor Physical Address

• IOMMU introduced in the system along-with IOV support on the I/O devices to support direct I/O access to the GuestOS

Virtualization - Pros and Cons

Advantages of Virtualization

Using Virtualization for Efficient Hardware Utilization

Virtualization decreases costs by reducing the need for physical hardware systems.

Virtual machines use efficient hardware, which lowers the quantities of hardware, associated maintenance costs and reduces the power along with cooling the demand.

You can allocate memory, space and CPU in just a second, making you more self-independent from hardware vendors.

Using Virtualization to Increase Availability

Virtualization platforms offer a number of advanced features that are not found on physical servers, which increase uptime and availability.

Although the vendor feature names may be different, they usually offer capabilities such as live migration, storage migration, fault tolerance, high availability and distributed resource scheduling.

These technologies keep virtual machines chugging along or give them the ability to recover from unplanned outages.

The ability to move a virtual machine from one server to another is perhaps one of the greatest single benefits of virtualization with far reaching uses.

As the technology continues to mature to the point where it can do long-distance migrations, such as being able to move a virtual machine from one data center to another no matter the network latency involved.

Disaster Recovery

Disaster recovery is very easy when your servers are virtualized.

With up-to-date snapshots of your virtual machines, you can quickly get back up and running.

An organization can more easily create an affordable replication site.

If a disaster strikes in the data center or server room itself, you can always move those virtual machines elsewhere into a cloud provider.

Having that level of flexibility means your disaster recovery plan will be easier to enact and will have a 99% success rate.

Save Energy

Moving physical servers to virtual machines and consolidating them onto far fewer physical servers’ means lowering monthly power and cooling costs in the data center.

It reduces carbon footprint and helps to clean up the air we breathe. Consumers want to see companies reducing their output of pollution and taking responsibility.

Deploying Servers too fast

You can quickly clone an image, master template or existing virtual machine to get a server up and running within minutes.

You do not have to fill out purchase orders, wait for shipping and receiving and then rack, stack, and cable a physical machine only to spend additional hours waiting for the operating system and applications to complete their installations.

With virtual backup tools like Veeam, redeploying images will be so fast that your end users will hardly notice there was an issue.

Save Space in your Server Room or Datacenter

Imagine a simple example: you have two racks with 30 physical servers and 4 switches.

By virtualizing your servers, it will help you to reduce half the space used by the physical servers.

The result can be two physical servers in a rack with one switch, where each physical server holds 15 virtualized servers.

Testing and setting up Lab Environment

While you are testing or installing something on your servers and it crashes, do not panic, as there is no data loss.

Just revert to a previous snapshot and you can move forward as if the mistake did not even happen.

You can also isolate these testing environments from end users while still keeping them online.

When you have completely done your work, deploy it in live.

Shifting all your Local Infrastructure to Cloud in a day

If you decide to shift your entire virtualized infrastructure into a cloud provider, you can do it in a day.

All the hypervisors offer you tools to export your virtual servers.

Possibility to Divide Services

If you have a single server, holding different applications this can increase the possibility of the services to crash with each other and increasing the fail rate of the server.

If you virtualize this server, you can put applications in separated environments from each other as we have discussed previously.

Disadvantages of Virtualization

Although you cannot find many disadvantages for virtualization, we will discuss a few prominent ones as follows −

Extra Costs

Maybe you have to invest in the virtualization software and possibly additional hardware might be required to make the virtualization possible.

This depends on your existing network.

Many businesses have sufficient capacity to accommodate the virtualization without requiring much cash.

If you have an infrastructure that is more than five years old, you have to consider an initial renewal budget.

Software Licensing

This is becoming less of a problem as more software vendors adapt to the increased adoption of virtualization.

However, it is important to check with your vendors to understand how they view software use in a virtualized environment.

Learn the new Infrastructure

Implementing and managing a virtualized environment will require IT staff with expertise in virtualization.

On the user side, a typical virtual environment will operate similarly to the non-virtual environment.
There are some applications that do not adapt well to the virtualized environment.

Best Data Visualization Tools Available Today

Big Data and the ever-growing access we have to more information is the driving force behind artificial intelligence and the wave of technological change sweeping across all industries.

But all the data in the world is useless – in fact it can become a liability – if you can’t understand it.

Data visualization is about how to present your data, to the right people, at the right time, in order to enable them to gain insights most effectively.

Luckily visualization solutions are evolving as rapidly as the rest of the tech stack. Charts, videos, infographics and at the cutting edge even virtual reality and augmented reality (VR & AR) presentations offer increasingly engaging and intuitive channels of communication.

Here’s my run-down of some of the best, most popular or most innovative data visualization tools available today. These are all paid-for (although they all offer free trials or personal-use licences).

Look out for another post soon on completely free and open source alternatives.

Tableau

Tableau is often regarded as the grand master of data visualization software and for good reason.

Tableau has a very large customer base of 57,000+ accounts across many industries due to its simplicity of use and ability to produce interactive visualizations far beyond those provided by general BI solutions.

It is particularly well suited to handling the huge and very fast-changing datasets which are used in Big Data operations, including artificial intelligence and machine learning applications, thanks to integration with a large number of advanced database solutions including Hadoop, Amazon AWS, My SQL, SAP and Teradata.

Extensive research and testing has gone into enabling Tableau to create graphics and visualizations as efficiently as possible, and to make them easy for humans to understand.

Qlikview

Qlik with their Qlikview tool is the other major player in this space and Tableau’s biggest competitor.

The vendor has over 40,000 customer accounts across over 100 countries, and those that use it frequently cite its highly customizable setup and wide feature range as a key advantage.
This however can mean that it takes more time to get to grips with and use it to its full potential.

In addition to its data visualization capabilities Qlikview offers powerful business intelligence, analytics and enterprise reporting capabilities and I particularly like the clean and clutter-free user interface.

Qlikview is commonly used alongside its sister package, Qliksense, which handles data exploration and discovery.

There is also a strong community and there are plenty of third-party resources available online to help new users understand how to integrate it in their projects.

FusionCharts

This is a very widely-used, JavaScript-based charting and visualization package that has established itself as one of the leaders in the paid-for market.

It can produce 90 different chart types and integrates with a large number of platforms and frameworks giving a great deal of flexibility.

One feature that has helped make FusionCharts very popular is that rather than having to start each new visualization from scratch, users can pick from a range of “live” example templates, simply plugging in their own data sources as needed.

Highcharts

Like FusionCharts this also requires a licence for commercial use, although it can be used freely as a trial, non-commercial or for personal use.

Its website claims that it is used by 72 of the world’s 100 largest companies and it is often chosen when a fast and flexible solution must be rolled out, with a minimum need for specialist data visualization training before it can be put to work.

A key to its success has been its focus on cross-browser support, meaning anyone can view and run its interactive visualizations, which is not always true with newer platforms.

Datawrapper

Datawrapper is increasingly becoming a popular choice, particularly among media organizations which frequently use it to create charts and present statistics.

It has a simple, clear interface that makes it very easy to upload csv data and create straightforward charts, and also maps, that can quickly be embedded into reports.

Plotly

Plotly enables more complex and sophisticated visualizations, thanks to its integration with analytics-oriented programming languages such as Python, R and Matlab.

It is built on top of the open source d3.js visualization libraries for JavaScript, but this commercial package (with a free non-commercial licence available) adds layers of user-friendliness and support as well as inbuilt support for APIs such as Salesforce.

Sisense

Sisense provides a full stack analytics platform but its visualization capabilities provide a simple-to-use drag and drop interface which allow charts and more complex graphics, as well as interactive visualizations, to be created with a minimum of hassle.

It enables multiple sources of data to be gathered into one easily accessed repositories where it can be queried through dashboards instantaneously, even across Big Data-sized sets.

Dashboards can then be shared across organizations ensuring even non technically-minded staff can find the answers they need to their problems.

⇨Data Mining

Data Mining is the set of methodologies used in analyzing data from various dimensions and perspectives, finding previously unknown hidden patterns, classifying and grouping the data and summarizing the identified relationships.

The tasks of data mining are twofold:

Create predictive power using features to predict unknown or future values of the same or other feature — and Create a descriptive power, find interesting, human-interpretable patterns that describe the data.

Four most useful data mining techniques:

➥Regression (predictive)
➥Association Rule Discovery (descriptive)
➥Classification (predictive)
➥Clustering (descriptive) and there are many more ……

Comprehensive List of tools for Data Mining

➯Rapid Miner
➯Weka
➯Orange
➯R
➯Knime
➯Rattle
➯Tanagra
➯XL Miner

Hypervisor

For the most part, cloud computing entails you being able to access a virtual machine for you to be able to do what you need to do anywhere.
A hypervisor manages these virtual machines.

A hypervisor is a program that would enable you to host several different virtual machines on a single hardware.
Each one of these virtual machines or operating systems you have will be able to run its own programs, as it will appear that the system has the host hardware's processor, memory and resources. In reality, however, it is actually the hypervisor that is allocating those resources to the virtual machines.

In effect, a hypervisor allows you to have several virtual machines all working optimally on a single piece of computer hardware.

You may think that the hypervisor is a fairly recent phenomenon.
The first hypervisors were introduced in the 1960s to allow for different operating systems on a single mainframe computer.
However, its current popularity is largely due to Linux and Unix. Around 2005, Linux and Unix systems started using virtualization technology to expand hardware capabilities, control costs, and improved reliability and security that hypervisors provided to these systems.

Now, hypervisors are fundamental components of any virtualization effort.
You can think of it as the operating system for virtualized systems.
It can access all physical devices residing on a server.
It can also access the memory and disk. It can control all aspects and parts of a virtual machine.

It work

The servers would need to execute the hypervisor.
The hypervisor, in turn, loads the client operating systems of the virtual machines.
The hypervisor allocates the correct CPU resources, memory, bandwidth and disk storage space for each virtual machine.

A virtual machine can create requests to the hypervisor through a variety of methods, including API calls.

There are two types of hypervisors:

🔻Embedded or hosted hypervisors, and
🔻Bare metal or native hypervisors.

Role of a Hypervisor

The explanation of a hypervisor up to this point has been fairly simple: it is a layer of software that sits between the hardware and the one or more virtual machines that it supports.
Its job is also fairly simple.
The three characteristics defined by Popek and Goldberg illustrate these tasks:

🍎Provide an environment identical to the physical environment
🍎Provide that environment with minimal performance cost
🍎Retain complete control of the system resources

Differences between an “Emulator” and a “Virtual Machine”

Virtual machines make use of CPU self-virtualization, to whatever extent it exists, to provide a virtualized interface to the real hardware.

Emulators emulate hardware without relying on the CPU being able to run code directly and redirect some operations to a hypervisor controlling the virtual container.

A specific x86 example might help: Bochs is an emulator, emulating an entire processor in software even when it's running on a compatible physical processor; qemu is also an emulator, although with the use of a kernel-side kqemu package it gained some limited virtualization capability when the emulated machine matched the physical hardware — but it could not really take advantage of full x86 self-virtualization, so it was a limited hypervisor; kvm is a virtual machine hypervisor.

A hypervisor could be said to "emulate" protected access; it doesn't emulate the processor, though, and it would be more correct to say that it mediates protected access.

Protected access means things like setting up page tables or reading/writing I/O ports.

For the former, a hypervisor validates (and usually modifies, to match the hypervisor's own memory) the page table operation and performs the protected instruction itself; I/O operations are mapped to emulated device hardware instead of emulated CPU.

And just to complicate things, Wine is also more a hypervisor/virtual machine (albeit at a higher ABI level) than an emulator (hence "Wine Is Not an Emulator").

Virtual Machine

A virtual machine (VM) is an emulation of a computer system.
Put simply, it makes it possible to run what appear to be many separate computers on hardware that is actually one computer.

The operating systems (“OS”) and their applications share hardware resources from a single host server, or from a pool of host servers.
Each VM requires its own underlying OS, and the hardware is virtualized. A hypervisor, or a virtual machine monitor, is software, firmware, or hardware that creates and runs VMs.
It sits between the hardware and the virtual machine and is necessary to virtualize the server.

Since the advent of affordable virtualization technology and cloud computing services, IT departments large and small have embraced virtual machines (VMs) as a way to lower costs and increase efficiencies.

VMs,however, can take up a lot of system resources.

Each VM runs not just a full copy of an operating system, but a virtual copy of all the hardware that the operating system needs to run.

This quickly adds up to a lot of RAM and CPU cycles.

That’s still economical compared to running separate actual computers, but for some applications it can be overkill, which led to the development of containers.

Benefits of VMs

All OS resources available to apps
Established management tools
Established security tools
Better known security controls
Popular VM Providers
VMware vSphere
VirtualBox
Xen
Hyper-V
KVM

Containers

With containers, instead of virtualizing the underlying computer like a virtual machine (VM), just the OS is virtualized.

Containers sit on top of a physical server and its host OS — typically Linux or Windows. Each container shares the host OS kernel and, usually, the binaries and libraries, too.

Shared components are read-only. Sharing OS resources such as libraries significantly reduces the need to reproduce the operating system code, and means that a server can run multiple workloads with a single operating system installation.

Containers are thus exceptionally light — they are only megabytes in size and take just seconds to start.

Compared to containers, VMs take minutes to run and are an order of magnitude larger than an equivalent container.

In contrast to VMs, all that a container requires is enough of an operating system, supporting programs and libraries, and system resources to run a specific program.

What this means in practice is you can put two to three times as many as applications on a single server with containers than you can with a VM.

In addition, with containers you can create a portable, consistent operating environment for development, testing, and deployment.

Types of Containers

Linux Containers (LXC) ➪The original Linux container technology is Linux Containers, commonly known as LXC.
LXC is a Linux operating system level virtualization method for running multiple isolated Linux systems on a single host.

Docker ➪Docker started as a project to build single-application LXC containers, introducing several changes to LXC that make containers more portable and flexible to use.
It later morphed into its own container runtime environment.
At a high level, Docker is a Linux utility that can efficiently create, ship, and run containers.

Benefits of Containers

Reduced IT management resources
Reduced size of snapshots
Quicker spinning up apps
Reduced & simplified security updates
Less code to transfer, migrate, upload workloads
Popular Container Providers
Linux Containers
LXC
LXD
CGManager
Docker
Windows Server Containers

Uses for VMs vs Uses for Containers

Both containers and VMs have benefits and drawbacks, and the ultimate decision will depend on your specific needs, but there are some general rules of thumb.

VMs are a better choice for running apps that require all of the operating system’s resources and functionality, when you need to run multiple applications on servers, or have a wide variety of operating systems to manage.

Containers are a better choice when your biggest priority is maximizing the number of applications running on a minimal number of servers.

Diff: VMs vs Containers

Git config levels and files

The git config command can accept arguments to specify which configuration level to operate on. The following configuration levels are available:

⛄Local

By default, git config will write to a local level if no configuration option is passed.
Local level configuration is applied to the context repository git config gets invoked in.
Local configuration values are stored in a file that can be found in the repo's .git directory: .git/config

⛄Global

Global level configuration is user-specific, meaning it is applied to an operating system user.
Global configuration values are stored in a file that is located in a user's home directory.
~ /.gitconfig on UNIX systems and C:\Users\<username>\.gitconfig on windows

There is 2 git config we can use, local & global.
If you have multiple git servers as me, you need to config your git as per folder bases. Or you can use the global config.

For local git repo-config, use:

git config user. name "your-user-name"
git config user.email "your-email-addr"

This will result in [user] section added to .git/config file:

[user]
name = your-user-name
email = your-email-address

You can use the global config also if you only have one git server.

git config --global user.name "your-user-name"
git config --global user.email "your-email-addr"

Then the [user] section will be present at ~/.gitconfig, with the same content as in the .git/config file.

Setting your username in Git

Git uses a username to associate commits with an identity.
The Git username is not the same as your GitHub username.

You can change the name that is associated with your Git commits using the git config command. The new name you set will be visible in any future commits you to push to GitHub from the command line.
If you'd like to keep your real name private, you can use any text as your Git username.

Changing the name associated with your Git commits using git config will only affect future commits and will not change the name used for past commits.

Setting your Git username for a single repository

1.Open Git Bash.

2. Change the current working directory to the local repository where you want to configure the name that is associated with your Git commits.

3.Set a Git username:

4.$ git config user.name "Linda Linsa"

Confirm that you have set the Git username correctly:

5.$ git config user.name

> Linda Linsa

What happens when I change my username?

When you change your GitHub username, most references to your repositories under the old username automatically change to the new username.

However, some links to your profile won't automatically redirect.

Commits made with your username@users.noreply.github.com email address will no longer be associated with your account.
Commits made using your ID+username@users.noreply.github.com email address will continue to be associated with your account.

Repository references

After you change your username, GitHub will automatically redirect references to your repositories.

Web links to your existing repositories will continue to work. This can take a few minutes to complete after you make the change.

Command line pushes from your local repository clones to the old remote tracking URLs will continue to work.

After changing your username, your old username becomes available for anyone else to claim. If the new owner of your old username creates a repository with the same name as your repository, that will override the redirect entry and your redirect will stop working. Because of this possibility, we recommend you update all existing remote repository URLs after changing your username. For more information, see "Changing a remote's URL."

GitHub cannot set up redirects for:

⇨@mentions using your old username
⇨Links to Gists with your old username

Links to your previous profile page

After changing your username, links to your previous profile page, such as https://github.com/previoususername, will return a 404 error.
We recommend updating any links to your GitHub account from elsewhere, such as your LinkedIn or Twitter profile.

Your Git commits

Git commits that were associated with your GitHub-provided no reply email address won't be attributed to your new username and won't appear in your contributions graph.
If your Git commits are associated with another email address you've added to your GitHub account, including the ID-based GitHub-provided no reply email address, they'll continue to be attributed to you and appear in your contributions graph after you've changed your username.
For more information on setting your email address, see "Setting your commit email address on GitHub."

Git Branch

This document is an in-depth review of the git branch command and a discussion of the overall Git branching model. Branching is a feature available in most modern version control systems.

Branching in other VCS's can be an expensive operation in both time and disk space. In Git, branches are a part of your everyday development process.

Git branches are effectively a pointer to a snapshot of your changes.

When you want to add a new feature or fix a bug—no matter how big or how small—you spawn a new branch to encapsulate your changes.

This makes it harder for an unstable code to get merged into the main code base, and it gives you the chance to clean up your future's history before merging it into the main branch.

The diagram above visualizes a repository with two isolated lines of development, one for a little feature, and one for a longer-running feature.
By developing them in branches, it’s not only possible to work on both of them in parallel, but it also keeps the main master branch free from questionable code.

The implementation behind Git branches is much more lightweight than other version control system models.

Instead of copying files from the directory to directory, Git stores a branch as a reference to a commit.
In this sense, a branch represents the tip of a series of commits—it's not a container for commits. The history for a branch is extrapolated through the commit relationships.

As you read, remember that Git branches aren't like SVN branches.

Whereas SVN branches are only used to capture the occasional large-scale development effort, Git branches are an integral part of your everyday workflow.

The following content will expand on the internal Git branching architecture.

Git 🌠 Adjustable commit message

Users utilizing the GIT connectors available in Memsource can now customize the commit message that will be displayed in their Git accounts once the translated text is pushed back to the online repository.

It is possible to use the following macros to let Memsource add specific parts of the message automatically:

⮞{sourceFileName}

⮞{projectName}

⮞{jobCreateTime}

Git init

This page will explore the git init command in depth. By the end of this page, you will be informed on the core functionality and extended feature set of git init.

This exploration includes:

git init options and usage
.git directory overview
custom git init directory environment values
git init vs. git clone
git init bare repositories
git init templates

The git init command creates a new Git repository.

It can be used to convert an existing, unversioned project to a Git repository or initialize a new, empty repository.

Most other Git commands are not available outside of an initialized repository, so this is usually the first command you'll run in a new project.

Executing git init creates a .git subdirectory in the current working directory, which contains all of the necessary Git metadata for the new repository.

This metadata includes subdirectories for objects, refs, and template files. A HEAD file is also created which points to the currently checked out commit.

Aside from the .git directory, in the root directory of the project, an existing project remains unaltered (unlike SVN, Git doesn't require a .git subdirectory in every subdirectory).

By default, git init will initialize the Git configuration to the .git subdirectory path. The subdirectory path can be modified and customized if you would like it to live elsewhere.

You can set the $GIT_DIR environment variable to a custom path and git init will initialize the Git configuration files there.

Additionally, you can pass the --separate-git-dir argument for the same result. A common use case for a separate .git sub directory is to keep your system configuration "dotfiles" (.bashrc, .vimrc, etc.) in the home directory while keeping the .git folder elsewhere.

Usage

Compared to SVN, the git init command is an incredibly easy way to create new version-controlled projects.

Git doesn’t require you to create a repository, import files, and check out a working copy.

Additionally, Git does not require any pre-existing server or admin privileges.

All you have to do is cd into your project sub directory and run git init, and you'll have a fully functional Git repository.

git init

Transform the current directory into a Git repository.

This adds a .git subdirectory to the current directory and makes it possible to start recording revisions of the project.

git init <directory>

Create an empty Git repository in the specified directory.

Running this command will create a new sub directory called containing nothing but the .git sub directory.

If you've already run git init on a project directory and it contains a .git sub directory, you can safely run git init again on the same project directory. It will not override an existing .git configuration.

Not recommended to have an Eclipse project folder as a Git repository

It is probably not a good idea to make a project the root folder of your Repository

The reason is that you will never be able to add another project to this Repository, as the .project file will occupy the root folder; you could still add projects as sub-folders, but this kind of project nesting is known to cause lots of problems all over the place.

In order to add another project, you would have to move the project to a sub-folder in the Repository and add the second project as another sub-folder before you could commit this change.

🌹Some more information

It is a good idea to keep your Repository outside of your Eclipse Work space

There are several reasons for this:

The new Repository will consider the complete folder structure of the Eclipse work space as (potential) content.

This can result in performance issues, for example when calculating the changes before committing (which will scan the complete .metadata folder, for example); more often than not, the work space will contain dead folders (e.g. deleted projects) which semantically are not relevant for E Git but can not be excluded easily.

The metadata (.git-) folder will be a child of the Eclipse Work space. It is unclear whether this might cause unwanted folder traversals by Eclipse.

You can easily destroy your Repository by destroying your Eclipse Work space.

Perspective

An eclipse perspective is the name given to an initial collection and arrangement of views and an editor area.

The default perspective is called java.

An eclipse window can have multiple perspectives open in it but only one perspective is active at any point of time.

A user can switch between open perspectives or open a new perspective.

The active perspective controls what appears in some menus and tool bars.

Opening a Perspective

To open a new perspective, click on the Windows menu and select Open Perspective → Other

The Open Perspective dialog box shows all the available perspectives. The same dialog can be brought up by clicking on the Open Perspective button (Perspective Button ) on the tool bar.

Switching between Perspectives

The most commonly used perspectives for java development are the Java perspective and Debug perspective.

Users can switch between open perspectives by clicking on the Perspective name on the toolbar.

Closing a Perspective

To close a perspective, right click on the perspective name in toolbar and select the Close menu item.

Customizing a Perspective

The customize perspective dialog can be used to customize a perspective.

Customizing a perspective means −

🎈Determining the icons visible on the toolbar when a perspective is active.
🎈Determining the menu items visible when a perspective is active.
🎈Determine the menu items in New sub menu, Show View sub menu and Open Perspective sub menu.

The tab descriptions of customize perspective are given below −

The Tool Bar Visibility tab can be used to determine which icons are visible on the toolbar when a perspective is open.

The Menu Visibility tab can be used to determine which menu items are visible when a perspective is active.

The Command Groups Availability tab can be used to control the visibility of toolbar icons and menu items.

The Shortcuts tab can be used to determine the menu items in New sub menu, Show View sub menu and Open Perspective sub menu.

Based on the Shortcuts selection in the picture given below, to bring up the "New Java Project from Existing Ant Build File" wizard users have to bring up the New sub menu (File → New), click on Other, expand the Java category and then select "Java Project from Existing Ant Build File".

On the other hand, to start the New Class wizard they can bring up the New sub menu (File → New) and select the Class menu item because its selected in the picture given below. By selecting the "Java Project from Existing Ant Build File" check box this item will also appear under the New menu.

Wrote by Hansi