• At the most basic level, an information system (IS) is a set of components that work together to manage data processing and storage. Its role is to support the key aspects of running an organization, such as communication, record-keeping, decision making, data analysis and more. Companies use this information to improve their business operations, make strategic decisions and gain a competitive edge.
• Information systems typically include a combination of software, hardware and telecommunication networks. For example, an organization may use customer relationship management systems to gain a better understanding of its target audience, acquire new customers and retain existing clients. This technology allows companies to gather and analyze sales activity data, define the exact target group of a marketing campaign and measure customer satisfaction.

• Modern technology can significantly boost your company's performance and productivity. Information systems are no exception. Organizations worldwide rely on them to research and develop new ways to generate revenue, engage customers and streamline time-consuming tasks.
• With an information system, businesses can save time and money while making smarter decisions. A company's internal departments, such as marketing and sales, can communicate better and share information more easily.
• Since this technology is automated and uses complex algorithms, it reduces human error. Furthermore, employees can focus on the core aspects of a business rather than spending hours collecting data, filling out paperwork and doing the manual analysis.
• Thanks to modern information systems, team members can access massive amounts of data from one platform. For example, they can gather and process information from different sources, such as vendors, customers, warehouses and sales agents, with a few mouse clicks.

• Persistent data is data that’s considered durable at rest with the coming and going of software and devices. Master data that’s stable—that is set and recoverable whether in flash or in memory.
• Here's what we heard when we asked,  "How do you define persistent data?":
• The opposite of dynamic—it doesn’t change and is not accessed very frequently.
• Core information, also known as dimensional information in data warehousing. Demographics of entities—customers, suppliers, orders.
• Master data that’s stable.
• Data that exists from one instance to another. Data that exists across time independent of the systems that created it. Now there’s always a secondary use for data, so there’s more persistent data. A persistent copy may be made or it may be aggregated. The idea of persistence is becoming more fluid.
• Stored in actual format and stays there versus in-memory where you have it once, close the file and it’s gone. You can retrieve persistent data again and again. Data that’s written to the disc; however, the speed of the discs is a bottleneck for the database. Trying to move to memory because it’s 16X faster.
• Every client has their own threshold for criticality (e.g. financial services don’t want to lose any debits or credits). Now, with much more data from machines and sensors, there is greater transactionality. The meta-data is as important as the data itself. Meta-data must be transactional.
• Non-volatile. Persists in the face of a power outage.
• Any data stored in a way that it stays stored for an extended period versus in-memory data. Stored in the system modeled and structured to endure power outages. Data doesn’t change at all.
• Data considered durable at rest with the coming and going of hardware and devices. There’s a persistence layer at which you hold your data at risk.
• Data that is set and recoverable whether in flash or memory backed.
• With persistent data, there is reasonable confidence that changes will not be lost and the data will be available later. Depending on the requirements, in-cloud or in-memory systems can qualify. We care most about the "data" part. If it’s data, we want to enable customers to read, query, transform, write, add-value, etc.
• A way to persist data to disk or storage. Multiple options to do so with one replica across data centers in any combination with and without persistence. Snapshot data to disk or snapshot changes. Write to disk every second or every write. Users can choose between all options. Persistence is part of a high availability suite which provides replication and instant failover. Registered over multiple clouds. Host thousands of instances over multiple data centers with only two node failures per day. Users can choose between multiple data centers and multiple geographies. We are the company behind Redis. Others treat as a cache and not a database. Multiple nodes - data written to disks. You can’t do that with regular open source. If you don’t do high availability, as recommended, you can lose your data.

Users access a database server either through a "front end" running on the user's computer – which displays requested data – or through the "back end", which runs on the server and handles tasks such as data analysis and storage.

DB-Engines lists over 300 DBMSs in its ranking.

Difference Between File and Database

File

A data file is a collection of related records stored on a storage medium such as a hard disk or optical disc. A Student file at a school might consist of thousands of individual student records. Each student record in the file contains the same fields. Each field, however, contains different data. The image shows a small sample Student file that contains four student records, each with eleven fields. A database includes a group of related data files.

Database

A database is a collection of data organized in a manner that allows access, retrieval, and use of that data. Data is a collection of unprocessed items, which can include text, numbers, images, audio, and video. Information is processed data; that is, it is organized, meaningful, and useful.

Computers process data in a database into information. A database at a school, for example, contains data about students, e.g., student data, class data, etc. A computer at the school processes new student data and then sends advising appointment and ID card information to the printers.

Database - Advantages & Disadvantages

   Advantages

• Reduced data redundancy
• Reduced updating errors and increased consistency
• Greater data integrity and independence from applications programs
• Improved data access to users through the use of host and query languages
• Improved data security
• Reduced data entry, storage, and retrieval costs
• Facilitated development of new applications program

   Disadvantages

• Database systems are complex, difficult and time-consuming to design
• Substantial hardware and software start-up costs
• Damage to database affects virtually all applications programs
• Extensive conversion costs in moving from a file-based system to a database system
• Initial training required for all programmers and users

File system - Advantages & Disadvantages

Advantages 

1. it supports heterogeneous operating systems including all flavors of the UNIX operating system as well as Linux and windows

2. multiple client machines can access a single resource simultaneously.

3. enables sharing of common application binaries and read-only information instead of putting them on every single machine. This results in reduced overall disk storage cost and administration overhead.

4. gives access to uniform data to groups of users.

5. useful when many users exist on many systems with each user's home directory located on every single machine. Network file systems allow you to all users home directories on a single machine under /home

Disadvantage

1. Program-Data Dependence. File descriptions are stored within each application program that accesses a given file.

2. Duplication of Data. Applications are developed independently in file processing systems leading to unplanned duplicate files. Duplication is wasteful as it requires additional storage space and changes in one file must be made manually in all files. This also results in a loss of data integrity. It is also possible that the same data item may have different names in different files, or the same name may be used for different data items in different files.

3. Limited data sharing. Each application has its own private files with little opportunity to share data outside their own applications. A requested report may require data from several incompatible files in separate systems.

4. Lengthy Development Times. There is little opportunity to leverage previous development efforts. Each new application requires the developer to start from scratch by designing new file formats and descriptions

5. Excessive Program Maintenance. The preceding factors create a heavy program maintenance load.

6. Integrity Problem. The problem of integrity is the problem of ensuring that the data in the database is accentuate

•  Big Data includes huge volume, high velocity, and extensible variety of data. These are 3 types: Structured data, Semi-structured data, and Unstructured data.

Structured Data 

Structured data is data whose elements are addressable for effective analysis. It has been organized into a formatted repository that is typically a database. It concerns all data which can be stored in database SQL in the table with rows and columns. They have a relational key and can easily be mapped into pre-designed fields. Today, those data are most processed in development and simplest way to manage information.

Example: Relational data.

Semi-Structured Data 

Semi-structured data is information that does not reside in a rational database but that have some organizational properties that make it easier to analyze. With some process, you can store them in the relation database (it could be very hard for some kind of semi-structured data), but Semi-structured exist to ease space.

Example: XML data.

Unstructured Data 

Unstructured data is a data that is which is not organized in a pre-defined manner or does not have a pre-defined data model, thus it is not a good fit for a mainstream relational database. So for Unstructured data, there are alternative platforms for storing and managing, it is increasingly prevalent in IT systems and is used by organizations in a variety of business intelligence and analytics applications.

 Example: Word, PDF, Text, Media logs.





Differences between Structured, Semi-structured and Unstructured data:

Types of Databases

1. Centralized Database

The information(data) is stored at a centralized location and the users from different locations can access this data. This type of database contains application procedures that help the users to access the data even from a remote location.

Various kinds of authentication procedures are applied for the verification and validation of end users, likewise, a registration number is provided by the application procedures which keeps a track and record of data usage. The local area office handles this thing.

2.Distributed Database

Just opposite of the centralized database concept, the distributed database has contributions from the common database as well as the information captured by local computers also. The data is not at one place and is distributed at various sites of an organization. These sites are connected to each other with the help of communication links which helps them to access the distributed data easily.

You can imagine a distributed database as a one in which various portions of a database are stored in multiple different locations(physical) along with the application procedures which are replicated and distributed among various points in a network.

There are two kinds of distributed database, viz. homogenous and heterogeneous. The databases which have the same underlying hardware and run over the same operating systems and application procedures are known as homogeneous DDB, for eg. All physical locations in a DDB. Whereas, the operating systems, underlying hardware as well as application procedures can be different at various sites of a DDB which is known as heterogeneous DDB.

3.Personal Database

Data is collected and stored on personal computers which are small and easily manageable. The data is generally used by the same department of an organization and is accessed by a small group of people.

4.End User Database

The end user is usually not concerned about the transaction or operations done at various levels and is only aware of the product which may be a software or an application. Therefore, this is a shared database which is specifically designed for the end user, just like different levels’ managers. Summary of whole information is collected in this database.

5.Commercial Database

These are the paid versions of the huge databases designed uniquely for the users who want to access the information for help. These databases are subject specific, and one cannot afford to maintain such huge information. Access to such databases is provided through commercial links.

6.NoSQL Database

These are used for large sets of distributed data. There are some big data performance issues which are effectively handled by relational databases, such kind of issues are easily managed by NoSQL databases. There are very efficient in analyzing large size unstructured data that may be stored at multiple virtual servers of the cloud.

7.Operational Database

Information related to operations of an enterprise is stored inside this database. Functional lines like marketing, employee relations, customer service etc. require such kind of databases.

 8.Relational Databases 

These databases are categorized by a set of tables where data gets fit into a pre-defined category. The table consists of rows and columns where the column has an entry for data for a specific category and rows contains instance for that data defined according to the category. The Structured Query Language (SQL) is the standard user and application program interface for a relational database.

There are various simple operations that can be applied over the table which makes these databases easier to extend, join two databases with a common relation and modify all existing applications.

9.Cloud Databases

Now a day, data has been specifically getting stored over clouds also known as a virtual environment, either in a hybrid cloud, public or private cloud. A cloud database is a database that has been optimized or built for such a virtualized environment. There are various benefits of a cloud database, some of which are the ability to pay for storage capacity and bandwidth on a per-user basis, and they provide scalability on demand, along with high availability.

A cloud database also gives enterprises the opportunity to support business applications in a software-as-a-service deployment

10.Object-Oriented Databases

An object-oriented database is a collection of object-oriented programming and relational database. There are various items which are created using object-oriented programming languages like C++, Java which can be stored in relational databases, but object-oriented databases are well-suited for those items.

An object-oriented database is organized around objects rather than actions, and data rather than logic. For example, a multimedia record in a relational database can be a definable data object, as opposed to an alphanumeric value.

11.Graph Databases

The graph is a collection of nodes and edges where each node is used to represent an entity and each edge describes the relationship between entities. A graph-oriented database, or graph database, is a type of NoSQL database that uses graph theory to store, map and query relationships.

Graph databases are basically used for analyzing interconnections. For example, companies might use a graph database to mine data about customers from social media.

Difference between Big Data and Data Warehouse

• Data Warehousing is one of the common words for the last 10-20 years, whereas Big Data is a hot trend for the last 5-10 years. Both of them hold a lot of data, used for reporting, managed by an electronic storage device. So one common thought of maximum people that recent big data will replace old data warehousing very soon. But still, big data and data warehousing is not interchangeable as they used totally for a different purpose. 

Application To Files/DB

• Files and DBs are external components
• They are existing outside the software system
• Software can connect to the files/DBs to perform CRUD operations on data

•File – File path, URL

   •DB – connection string

• To process data in DB

•SQL statements

      •Prepared statements

    •Callable statements

Statements, PreparedStatement and CallableStatement

Once a connection is obtained we can interact with the database. The JDBC Statement, CallableStatement, and PreparedStatement interfaces define the methods and properties that enable you to send SQL or PL/SQL commands and receive data from your database.

They also define methods that help bridge data type differences between Java and SQL data types used in a database.

The following table provides a summary of each interface's purpose to decide on the interface to use.

The Statement Objects

• Creating Statement Object

Before you can use a Statement object to execute a SQL statement, you need to create one using the Connection object's createStatement( ) method, as in the following example −

Statement stmt = null;
try {
   stmt = conn.createStatement( );
   . . .
}
catch (SQLException e) {
   . . .
}
finally {
   . . .
}

Once you've created a Statement object, you can then use it to execute an SQL statement with one of its three execute methods.

• boolean execute (String SQL): Returns a boolean value of true if a ResultSet object can be retrieved; otherwise, it returns false. Use this method to execute SQL DDL statements or when you need to use truly dynamic SQL.

• int executeUpdate (String SQL): Returns the number of rows affected by the execution of the SQL statement. Use this method to execute SQL statements for which you expect to get a number of rows affected - for example, an INSERT, UPDATE, or DELETE statement.

• ResultSet executeQuery (String SQL): Returns a ResultSet object. Use this method when you expect to get a result set, as you would with a SELECT statement.

Closing Statement Object

Just as you close a Connection object to save database resources, for the same reason you should also close the Statement object.

A simple call to the close() method will do the job. If you close the Connection object first, it will close the Statement object as well. However, you should always explicitly close the Statement object to ensure proper cleanup.

Statement stmt = null;
try {
   stmt = conn.createStatement( );
   . . .
}
catch (SQLException e) {
   . . .
}
finally {
   stmt.close();
}

For a better understanding, we suggest you study the Statement - Example tutorial.

The PreparedStatement Objects

The PreparedStatement interface extends the Statement interface, which gives you added functionality with a couple of advantages over a generic Statement object.

This statement gives you the flexibility of supplying arguments dynamically.

• Creating PreparedStatement Object

PreparedStatement pstmt = null;
try {
   String SQL = "Update Employees SET age = ? WHERE id = ?";
   pstmt = conn.prepareStatement(SQL);
   . . .
}
catch (SQLException e) {
   . . .
}
finally {
   . . .
}

All parameters in JDBC are represented by the? symbol, which is known as the parameter marker. You must supply values for every parameter before executing the SQL statement.

The setXXX() methods bind values to the parameters, where XXX represents the Java data type of the value you wish to bind to the input parameter. If you forget to supply the values, you will receive an SQLException.

Each parameter marker is referred by its ordinal position. The first marker represents position 1, the next position 2, and so forth. This method differs from that of Java array indices, which starts at 0.

All of the Statement object's methods for interacting with the database (a) execute(), (b) executeQuery(), and (c) executeUpdate() also work with the PreparedStatement object. However, the methods are modified to use SQL statements that can input the parameters.

• Closing PreparedStatement Object

Just as you close a Statement object, for the same reason you should also close the PreparedStatement object.

A simple call to the close() method will do the job. If you close the Connection object first, it will close the PreparedStatement object as well. However, you should always explicitly close the PreparedStatement object to ensure proper cleanup.

PreparedStatement pstmt = null;
try {
   String SQL = "Update Employees SET age = ? WHERE id = ?";
   pstmt = conn.prepareStatement(SQL);
   . . .
}
catch (SQLException e) {
   . . .
}
finally {
   pstmt.close();
}

For a better understanding, let us study Prepare - Example Code.

The CallableStatement Objects

Just as a Connection object creates the Statement and PreparedStatement objects, it also creates the CallableStatement object, which would be used to execute a call to a database stored procedure.

• Creating CallableStatement Object

Suppose, you need to execute the following Oracle stored procedure −

CREATE OR REPLACE PROCEDURE getEmpName 
   (EMP_ID IN NUMBER, EMP_FIRST OUT VARCHAR) AS
BEGIN
   SELECT first INTO EMP_FIRST
   FROM Employees
   WHERE ID = EMP_ID;
END;


NOTE: Above stored procedure has been written for Oracle, but we are working with MySQL database so, let us write a same stored procedure for MySQL as follows to create it in EMP database −

DELIMITER $$

DROP PROCEDURE IF EXISTS `EMP`.`getEmpName` $$
CREATE PROCEDURE `EMP`.`getEmpName` 
   (IN EMP_ID INT, OUT EMP_FIRST VARCHAR(255))
BEGIN
   SELECT first INTO EMP_FIRST
   FROM Employees
   WHERE ID = EMP_ID;
END $$

DELIMITER ;


Three types of parameters exist: IN, OUT, and INOUT. The PreparedStatement object only uses the IN parameter. The CallableStatement object can use all the three.

Here are the definitions of each −

The following code snippet shows how to employ the Connection.prepareCall() method to instantiate a CallableStatement object based on the preceding stored procedure −


CallableStatement cstmt = null;
try {
   String SQL = "{call getEmpName (?, ?)}";
   cstmt = conn.prepareCall (SQL);
   . . .
}
catch (SQLException e) {
   . . .
}
finally {
   . . .
}


The String variable SQL, represents the stored procedure, with parameter placeholders.

Using the CallableStatement objects is much like using PreparedStatement objects. You must bind values to all the parameters before executing the statement, or you will receive an SQLException.

If you have IN parameters, just follow the same rules and techniques that apply to a PreparedStatement object; use the setXXX() method that corresponds to the Java data type you are binding.

When you use OUT and INOUT parameters you must employ an additional CallableStatement method, registerOutParameter(). The registerOutParameter() method binds the JDBC data type, to the data type that the stored procedure is expected to return.

Once you call your stored procedure, you retrieve the value from the OUT parameter with the appropriate getXXX() method. This method casts the retrieved value of SQL type to a Java data type.

• Closing CallableStatement Object

Just as you close other Statement objects, for the same reason you should also close the CallableStatement object.

A simple call to the close() method will do the job. If you close the Connection object first, it will close the CallableStatement object as well. However, you should always explicitly close the CallableStatement object to ensure proper cleanup.

CallableStatement cstmt = null;
try {
   String SQL = "{call getEmpName (?, ?)}";
   cstmt = conn.prepareCall (SQL);
   . . .
}
catch (SQLException e) {
   . . .
}
finally {
   cstmt.close();
}


For a better understanding, I would suggest studying Callable - Example Code.

 ORM

 Pros 

• You get to write in the language you are already using anyway. If we’re being honest, we probably aren’t the greatest at writing SQL statements. SQL is a ridiculously powerful language, but most of us don’t write in it often. We do, however, tend to be much more fluent in one language or another and being able to leverage that fluency is awesome!
• It abstracts away the database system so that switching from MySQL to PostgreSQL, or whatever flavor you prefer, is easy-peasy.
• Depending on the ORM you get a lot of advanced features out of the box, such as support for transactions, connection pooling, migrations, seeds, streams, and all sorts of other goodies.
• Many of the queries you write will perform better than if you wrote them yourself.

 Cons 

• If you are a master at SQL, you can probably get more performant queries by writing them yourself.
• There is overhead involved in learning how to use any given ORM.
• The initial configuration of an ORM can be a headache.
• As a developer, it is important to understand what is happening under the hood. Since ORMs can serve as a crutch to avoid understanding databases and SQL, it can make you a weaker developer in that portion of the stack.

Popular ORMs

• If you’re going to use ORM, you should make your model objects as simple as possible. Be more vigilant about simplicity to make sure your model objects really are just Plain Ol’ Data. Otherwise, you may end up wrestling with your ORM to make sure the persistence works like you expect it to, and it’s not looking for methods and properties that aren’t actually there.
• If you’re not going to use ORM, you should probably define DAOs or persistence and query methods to avoid coupling the model layer with the persistence layer. Otherwise, you end up with SQL in your model objects and forced dependency on your project.
• If you know your data access patterns are generally going to be simple (like basic object retrieval) but you don’t know all of them up front, you should think about using an ORM. While ORMs can make building complex queries confusing to build and difficult to debug, an ORM can save you huge amounts of time if your queries are generally pretty simple.
• If you know your data access pattern is going to be complex or you plan to use a lot of database-specific features, you may not want to use an ORM. While many ORMs (like Hibernate) let you access the underlying data source connection pretty easily , if you know you’re going to have to throw around a lot of custom SQL, you may not get a lot of value out of ORM to begin with because you’re constantly going to have to break out of it.
• If it absolutely, positively, has to, has to, has to go fast, you may not want to use ORM. The only way to be absolutely sure all your queries consistently go fast is to plan your database structure carefully, manage your data access pattern with extreme prejudice, commit to one data store, and write your own queries optimized against that data store.
• Having mentioned some of the scenarios depending on which you could decide on whether or not to go with ORM, let me also point out a few Pros and Cons of ORM in general.

POJO 

Java Beans

POJO vs Java Bean

JPA

Where to use JPA?

JPA Providers

 .NET ORM Tools

•  .NET database connectivity tools with ORM support. It includes the following ORM tools and solutions:

•    Entity Developer is a powerful modeling and code generation tool for LinqConnect, Telerik Data Access, NHibernate, and ADO.NET Entity Framework.

• You can design an entity model from scratch or reverse-engineer an existing database. The model is used to generate C# or Visual Basic code with predefined or custom code templates.

• LINQ Insight is a powerful Visual Studio add-in for LINQ development that allows you to execute LINQ queries at design time directly from Visual Studio without starting a debug session and provides a powerful ORM profiler for Entity Framework, NHibernate, LINQ to SQL, and LinqConnect. 
• It profiles the data access layer of your projects and tracks all the ORM calls and SQL queries from the ORM.

• LinqConnect - a fast and easy to use ORM solution, developed closely to the Microsoft LINQ to SQL technology. In addition to LINQ to SQL features, LinqConnect provides its own advanced functionality. LinqConnect supports SQL Server, Oracle, MySQL, PostgreSQL, and SQLite.

Java ORM Tools

Hibernate

Hibernate is an object-relational mapping (ORM) library for the Java language, providing a framework for mapping an object-oriented domain model to a traditional relational database. Hibernate solves object-relational impedance mismatch problems by replacing direct persistence-related database accesses with high-level object handling functions.

IBatis / MyBatis

Toplink

In computing, TopLink is an object-relational mapping (ORM) package for Java developers. It provides a framework for storing Java objects in a relational database or for converting Java objects to XML documents.

PHP ORM Tools

• Doctrine ORM
• RedBeanPHP
• Eloquent ORM
• Propel
• Analogue ORM
• Sheep Orm
• RedBeanPHP4

NoSQL

NoSQL Database Types

The Benefits of NoSQL

 Hadoop

The 4 Modules of Hadoop

1. Distributed File-System

2. MapReduce

3. Hadoop Common

4. YARN

How Hadoop Came About

The Usage of Hadoop

Hadoop Architecture

Hadoop Daemons

Hadoop Flavors

Hadoop Ecosystem

Information Retrieval System

Definition:

The function of Information Retrieval:

Techniques of Information retrieval:

Basic Retrieval Tools

• Bibliographies
• Catalogs
• Indexes
• Finding Aids
• Registers
• Online Databases