Hicasct.files.wordpress.com
An Introduction to Open Source Software
Open source software is free software for your business or personal use. Open source developers freely share their knowledge and make the source code available to the public. The software is distributed with a license which allows other developers can modify it and/or add to it. Some examples of open source software are: WordPress, Ubuntu, and Mozilla, creators of the Firefox browser.
Advantages
• Open source software allows you to make choices
• Open source software is under constant development which addresses vulnerabilities, bug fixes, enhancements, and more.
• You can modify the software as necessary for your own purposes.
• Some open source programs give the user the option of automatic updates, which keep the software current. i.e. WordPress.
• A number of open source programs have a core application which can be enhanced by the use of plug-ins and themes. i.e. WordPress and Joomla.
• Open source software offers a tremendous amount of flexibility.
• Open source software is potentially more secure than commercial programs because the code is constantly being scrutinized by many programmers, not just a select few.
• Many open source programs can be installed on your computer, unlike a proprietary system which you can use, but where you have no control. If the software owner doesn’t like what you are doing they can wipe out your hard work overnight.
Disadvantages
• If you don´t know how to write code , you have to pay for modifications or learn how to code yourself.
• If the author of a product no longer supports the software you might be out of luck unless the development is picked up by other programmers You can take over development yourself if you wish.
• Open source is sometimes referred to as ‘open wallet’ in the sense that it may cost you more to have open source code modified than it would cost you to buy a commercial program.
• Unless there’s a structure in place to ensure the quality of the code it might wind up with many changes, bug fixes, and patches, all of which can make the code more complex and/or degrade the quality, which in turn leads to more maintenance.
• The software might not be well-documented, which could make it difficult to learn.
• Vulnerabilities in the software can be exploited by hackers. Another option is to make sure you have backups. This will save you a lot of time if you need to restore the site.
Difference Between Commercial and Open Source
|S.No. |Commercial |Open source |
|1 |Commercial systems are created and supported by for-profit |Open source systems are overseen by dedicated communities of developers |
| |companies (e.g., Microsoft) that typically sell licenses |who contribute modifications to improve the product continually and who |
| |for the use of their software and that are driven by |decide on the course of the software based on the needs of the community. |
| |maximizing profits. | |
|2 |Commercial software, on the other hand, requires purchasing|Open source software is generally free or has low-cost licensing options. |
| |a license. The up-front license cost of a commercial CMS | |
| |could run from a few thousand dollars to tens or even | |
| |hundreds of thousands. | |
|3 |Commercial or proprietary software also equates to better |While open source solutions are supported by communities of volunteers, |
| |support and typically offers a robust suite of features |your initial cost may be lower with this choice, but you will most likely |
| |right out of the box. If your organization’s needs are very|need to budget for technical resources to maintain it over time. With a |
| |well planned and documented, your IT favors |limited budget, however, your financial resources are better directed |
| |Microsoft products and commercially supported software, and|toward the best possible website as opposed to acquiring licenses and |
| |the up-front budget for software licensing is not a |paying mandatory fees for updates. |
| |significant concern, then Microsoft web stack and | |
| |commercial CMS may be a good option for you. | |
|4 |It includes a lot of extra features in our commercial |It provides a full package in the open source version but include a more |
| |version. So you will find that the commercial version does|limiting license. |
| |more (printing, text search, extraction) and it does things| |
| |better | |
|5 |They get money via |They get money via |
| |Product Sales |Consulting Sales |
| |Product Licenses |Support Contracts |
| |Product Renewals |SaaS - Software as a service / Hosting |
| |SaaS - Software as service / Hosting |Donations |
| |Consulting Sales | |
| |Support Contracts | |
| |Venture Capital | |
|6 |They market via |They market via |
| |Sales Team |Search Engine |
| |Marketing Team |Word of Mouth - Viral Marketing |
| |Advertising Dollars |Case Studies |
| |Search Engine | |
| |Word of Mouth - Viral Marketing | |
| |Case Studies | |
What is Linux?
• Linux is the best-known and most-used open source operating system. As an operating system, Linux is software that sits underneath all of the other software on a computer, receiving requests from those programs and relaying these requests to the computer’s hardware.
• Linux is an Open Source version of Unix developed by Linus Torvalds in 1991 to port Unix to the Intel x86 processor .This made Unix available on the most ubiquitous computer hardware that has ever existed, and therefore available to almost everyone.
• Linux has since been ported to almost every processor and function one could imagine, including game-boxes, personal digital assistants (PDAs), personal digital video recorders, and IBM mainframes, expanding the original concept of Unix for x86 to Unix for everything.
• For the purposes of this page, we use the term “Linux” to refer to the Linux kernel, but also the set of programs, tools, and services that are typically bundled together with the Linux kernel to provide all of the necessary components of a fully functional operating system.
• Some people, particularly members of the Free Software Foundation, refer to this collection as GNU/Linux, because many of the tools included are GNU components. However, not all Linux installations use GNU components as a part of their operating system. Android, for example, uses a Linux kernel but relies very little on GNU tools.
• Linux has a graphical interface, and types of software you are accustomed to using on other operating systems, such as word processing applications, have Linux equivalents.
• Linux is different from other operating systems in many important ways. Linux is open source software. The code used to create Linux is free and available to the public to view, edit, and for users with the appropriate skills—to contribute to.
• Linux is also different in that, although the core pieces of the Linux operating system are generally common, there are many distributions of Linux, which include different software options. This means that Linux is incredibly customizable, because not just applications, such as word processors and web browsers, can be swapped out. Linux users also can choose core components, such as which system displays graphics, and other user-interface components.
1 Usage of Linux
Companies and individuals choose Linux for their servers because it is secure, and you can receive excellent support from a large community of users, in addition to companies like Canonical, SUSE, and Red Hat, which offer commercial support.
Many of the devices you own probably, such as Android phones, digital storage devices, personal video recorders, cameras, wearables, and more, also run Linux. Even your car has Linux running under the hood.
2 Owners of Linux
Trademark on the name “Linux” rests with its creator, Linus Torvalds. The source code for Linux is under copyright by its many individual authors, and licensed under the GPLv2 license.
3 Contribution to Linux
Linux community is much more than the kernel, and needs contributions from lots of other people besides programmers. Every distribution contains hundreds or thousands of programs that can be distributed along with it, and each of these programs, as well as the distribution itself, need a variety of people and skill sets to make them successful, including:
• Testers to make sure everything works on different configurations of hardware and software, and to report the bugs when it does not.
• Designers to create user interfaces and graphics distributed with various programs.
• Writers who can create documentation, how-tos, and other important text distributed with software.
• Translators to take programs and documentation from their native languages and make them accessible to people around the world.
• Packagers to take software programs and put all the parts together to make sure they run flawlessly in different distributions.
• Evangelists to spread the word about Linux and open source in general.
• And of course developers to write the software itself.
4 Advantages and Benefits of Linux
• One of the significant benefits of open source software such as Linux is that because it has no owner, it can be debugged without resource to a license owner or software proprietor.
• The major advantage of Linux is its cost: the core OS is free, while many software applications also come with a GNU General public License. It can also be used simultaneously by large numbers of users without slowing down or freezing and it is very fast.
• It is an excellent networking platform and performs at optimum efficiency even with little available hard disk space.
• Linux also runs on a wide range of hardware types, including PCs, Macs, mainframes, supercomputers, some cell phones and industrial robots. Some prefer to dual-boot Linux and Windows while others prefer Linux and Mac OS. System76 machines come pre-installed with Linux in the form of Ubuntu, a Debian distribution of Linux. This is the most popular distribution of Linux for laptops
Benefits and advantages of Linux over other operating systems
• It is free to use and distribute.
• Support is free through online help sites, blogs and forums.
• It is very reliable – more so than most other operating systems with very few crashes.
• A huge amount of free open source software has been developed for it.
• It is very resistant to malware such as spyware, adware and viruses.
• It runs in a wide variety of machines than cannot be updated to use newer Windows versions.
• Since the source code is visible, ‘backdoors’ are easily spotted, so Linux offers greater security for sensitive applications.
• Linux offers a high degree of flexibility of configuration, and significant customization is possible without modifying the source code.
Free Softwares
• Linux is built with a collaborative development model. The operating system and most of its software are created by volunteers and employees of companies, governments and organisations from all over the world. The operating system is free to use and everyone has the freedom to contribute to its development. This co-operative development model means that everyone can benefit.
• Because of this, we like to call it Free Software, or Socially Responsible Software. Closely related is the concept of Open Source Software. Together, Free and Open Source Software is collectively abbreviated as FOSS.
• Transparency of the code and development process means that it can be participated in and audited at all levels.
• Linux has many other benefits, including speed, security and stability. It is renowned for its ability to run well on more modest hardware. Hence, viruses, worms, spyware and adware are basically a non-issue on Linux.
• Many FOSS developers develop for fun; many others are paid for their time. Because the code is open, it is actively worked on by all sorts of individuals and organisations. Since development is shared, it can cost relatively little to work with FOSS.
• When access to the source code is available, there are essentially no limitations to what can be achieved. Free Software is so named because of the freedom granted to the user.
• FOSS allows people and organisations to do what they want with the computers that they own, without being beholden to any company. They can make whatever modifications that they wish, providing unparalleled flexibility.
• Many groups in the government, business and education sectors use Linux as a means of cutting costs. It also allows them to create products that they would not otherwise be able to make.
• Schools both nationally and internationally are seeing the benefits of FOSS. There is a vast wealth of free software designed for children of all ages, including educational programmes and games. Education is all about imparting knowledge in an open fashion.
• Jimmy Wales, founder and leader of the Wikipedia project, explains, free knowledge cannot exist unless the tools used to manage it are also free.
• There are over 30 million users of Linux, and that number is growing rapidly. The Mozilla Firefox Web browser is the most popular Web browser and other open source based Web browsers such as Chrome, Safari and Konqueror.
• Linux and FOSS are major players in most other markets. The amazing flexibility and scalability of the software means that Linux can be found in computers both large and small.
1. Linux powers over 85 per cent of the top 500 supercomputers in the world, while also scaling down to run on one quarter of new smartphones.
2. Over 95 per cent of the servers and desktops at large animation and visual effects companies use Linux.
3. Linux drives over half of all Web servers, including 8 of the 10 most reliable hosting providers. The Apache Web server, a flagship example of FOSS, propels over 60 per cent of Web sites, including 44 per cent of secure (SSL) sites.
4. The One Laptop Per Child programme, a unique and ambitious collaboration between the United Nations and a multitude of governments, companies and other organisations worldwide.
Linux distributions tend to have their own support resources as well:
• for Ubuntu (and derivatives like Kubuntu, Xubuntu, Edubuntu, etc.)
1. Canonical's free technical support page
2. The Ubuntu Community
3. Community documentation
4. Unofficial Ubuntu Guide
• for Fedora
1. Fedora documentation
2. Communicating and Getting Help
3. Unofficial Fedora Guide
• for other distributions, consult the Web site of the project, as well as its DistroWatch page
Free Open Source Software (FOSS), also called just Open Source or Free Software, is licensed to be free to use, modify, and distribute. Most FOSS licenses also include a kind of legal Golden Rule, requiring any changes - such as fixes and enhancements - be released under the same license. This creates the trust in developers and users that generates large, sustainable communities that grow the software over time
The term free software refers to a lack of restrictions on individual users as well as zero cost; the term open source software refers to collaborative or networked development. FOSS, which embraces the benefits and adherents of both paradigms, is gaining widespread acceptance as traditional modes of software design are challenged.
The increasing popularity of FOSS has led to frustration in some circles for at least three reasons:
• Conventional software developers, distributors and sellers fear that FOSS will undercut their profits.
• Abuse of FOSS privileges may lead to questionable claims of copyright or trademark protection, thereby spawning litigation.
• The monetary value of FOSS is unclear, so governments have trouble figuring out how to tax it.
1 Why “Free Software” is better than “Open Source”
This article has been superseded by a major rewrite, “Open Source” misses the point of Free Software, which is much better. We keep this version for historical reasons.
While free software by any other name would give you the same freedom, it makes a big difference which name we use: different words convey different ideas.
In 1998, some of the people in the free software community began using the term “open source software” instead of “free software” to describe what they do. The term “open source” quickly became associated with a different approach, a different philosophy, different values, and even a different criterion for which licenses are acceptable. The Free Software movement and the Open Source movement are today separate movements with different views and goals, although we can and do work together on some practical projects.
The fundamental difference between the two movements is in their values, their ways of looking at the world. For the Open Source movement, the issue of whether software should be open source is a practical question, not an ethical one. As one person put it, “Open source is a development methodology; free software is a social movement.” For the Open Source movement, non-free software is a suboptimal solution. For the Free Software movement, non-free software is a social problem and free software is the solution.
1 Relationship between the Free Software movement and Open Source movement
The Free Software movement and the Open Source movement are like two political camps within the free software community.
Radical groups in the 1960s developed a reputation for factionalism: organizations split because of disagreements on details of strategy, and then treated each other as enemies. Or at least, such is the image people have of them, whether or not it was true.
The relationship between the Free Software movement and the Open Source movement is just the opposite of that picture. We disagree on the basic principles, but agree more or less on the practical recommendations. So we can and do work together on many specific projects. We don't think of the Open Source movement as an enemy. The enemy is proprietary software.
We are not against the Open Source movement, but we don't want to be lumped in with them. We acknowledge that they have contributed to our community, but we created this community, and we want people to know this. We want people to associate our achievements with our values and our philosophy, not with theirs. We want to be heard, not obscured behind a group with different views. To prevent people from thinking we are part of them, we take pains to avoid using the word “open” to describe free software, or its contrary, “closed”, in talking about non-free software.
So please mention the Free Software movement when you talk about the work we have done, and the software we have developed—such as the GNU/Linux operating system.
2 Comparing the two terms
This rest of this article compares the two terms “free software” and “open source”. It shows why the term “open source” does not solve any problems, and in fact creates some.
3 Ambiguity
The term “free software” has an ambiguity problem: an unintended meaning, “Software you can get for zero price,” fits the term just as well as the intended meaning, “software which gives the user certain freedoms.” We address this problem by publishing a more precise definition of free software, but this is not a perfect solution; it cannot completely eliminate the problem. An unambiguously correct term would be better, if it didn't have other problems.
Unfortunately, all the alternatives in English have problems of their own. We've looked at many alternatives that people have suggested, but none is so clearly “right” that switching to it would be a good idea. Every proposed replacement for “free software” has a similar kind of semantic problem, or worse—and this includes “open source software.”
The official definition of “open source software,” as published by the Open Source Initiative, is very close to our definition of free software; however, it is a little looser in some respects, and they have accepted a few licenses that we consider unacceptably restrictive of the users. However, the obvious meaning for the expression “open source software” is “You can look at the source code.” This is a much weaker criterion than free software; it includes free software, but also some proprietary programs, including Xv, and Qt under its original license (before the QPL).
That obvious meaning for “open source” is not the meaning that its advocates intend. The result is that most people misunderstand what those advocates are advocating. Here is how writer Neal Stephenson defined “open source”:
Linux is “open source” software meaning, simply, that anyone can get copies of its source code files.
I don't think he deliberately sought to reject or dispute the “official” definition. I think he simply applied the conventions of the English language to come up with a meaning for the term. The state of Kansas published a similar definition:
Make use of open-source software (OSS). OSS is software for which the source code is freely and publicly available, though the specific licensing agreements vary as to what one is allowed to do with that code.
Of course, the open source people have tried to deal with this by publishing a precise definition for the term, just as we have done for “free software.”
But the explanation for “free software” is simple—a person who has grasped the idea of “free speech, not free beer” will not get it wrong again. There is no such succinct way to explain the official meaning of “open source” and show clearly why the natural definition is the wrong one.
4 Fear of Freedom
The main argument for the term “open source software” is that “free software” makes some people uneasy. That's true: talking about freedom, about ethical issues, about responsibilities as well as convenience, is asking people to think about things they might rather ignore. This can trigger discomfort, and some people may reject the idea for that. It does not follow that society would be better off if we stop talking about these things.
Years ago, free software developers noticed this discomfort reaction, and some started exploring an approach for avoiding it. They figured that by keeping quiet about ethics and freedom, and talking only about the immediate practical benefits of certain free software, they might be able to “sell” the software more effectively to certain users, especially business. The term “open source” is offered as a way of doing more of this—a way to be “more acceptable to business.” The views and values of the Open Source movement stem from this decision.
This approach has proved effective, in its own terms. Today many people are switching to free software for purely practical reasons. That is good, as far as it goes, but that isn't all we need to do! Attracting users to free software is not the whole job, just the first step.
Sooner or later these users will be invited to switch back to proprietary software for some practical advantage. Countless companies seek to offer such temptation, and why would users decline? Only if they have learned to value the freedom free software gives them, for its own sake. It is up to us to spread this idea—and in order to do that, we have to talk about freedom. A certain amount of the “keep quiet” approach to business can be useful for the community, but we must have plenty of freedom talk too.
At present, we have plenty of “keep quiet”, but not enough freedom talk. Most people involved with free software say little about freedom—usually because they seek to be “more acceptable to business.” Software distributors especially show this pattern. Some GNU/Linux operating system distributions add proprietary packages to the basic free system, and they invite users to consider this an advantage, rather than a step backwards from freedom.
We are failing to keep up with the influx of free software users, failing to teach people about freedom and our community as fast as they enter it. This is why non-free software (which Qt was when it first became popular), and partially non-free operating system distributions, find such fertile ground. To stop using the word “free” now would be a mistake; we need more, not less, talk about freedom.
If those using the term “open source” draw more users into our community, that is a contribution, but the rest of us will have to work even harder to bring the issue of freedom to those users' attention. We have to say, “It's free software and it gives you freedom!”—more and louder than ever before.
5 Would a Trademark Help?
The advocates of “open source software” tried to make it a trademark, saying this would enable them to prevent misuse. This initiative was later dropped, the term being too descriptive to qualify as a trademark; thus, the legal status of “open source” is the same as that of “free software”: there is no legal constraint on using it. I have heard reports of a number of companies' calling software packages “open source” even though they did not fit the official definition; I have observed some instances myself.
But would it have made a big difference to use a term that is a trademark? Not necessarily.
Companies also made announcements that give the impression that a program is “open source software” without explicitly saying so. For example, one IBM announcement, about a program that did not fit the official definition, said this:
As is common in the open source community, users of the ... technology will also be able to collaborate with IBM ...
This did not actually say that the program was “open source”, but many readers did not notice that detail. (I should note that IBM was sincerely trying to make this program free software, and later adopted a new license which does make it free software and “open source”; but when that announcement was made, the program did not qualify as either one.)
And here is how Cygnus Solutions, which was formed to be a free software company and subsequently branched out (so to speak) into proprietary software, advertised some proprietary software products:
Cygnus Solutions is a leader in the open source market and has just launched two products into the [GNU/]Linux marketplace.
Unlike IBM, Cygnus was not trying to make these packages free software, and the packages did not come close to qualifying. But Cygnus didn't actually say that these are “open source software”, they just made use of the term to give careless readers that impression.
These observations suggest that a trademark would not have truly prevented the confusion that comes with the term “open source”.
6 Misunderstandings(?) of “Open Source”
The Open Source Definition is clear enough, and it is quite clear that the typical non-free program does not qualify. So you would think that “Open Source company” would mean one whose products are free software (or close to it), right? Alas, many companies are trying to give it a different meaning.
At the “Open Source Developers Day” meeting in August 1998, several of the commercial developers invited said they intend to make only a part of their work free software (or “open source”). The focus of their business is on developing proprietary add-ons (software or manuals) to sell to the users of this free software. They ask us to regard this as legitimate, as part of our community, because some of the money is donated to free software development.
In effect, these companies seek to gain the favorable cachet of “open source” for their proprietary software products—even though those are not “open source software”—because they have some relationship to free software or because the same company also maintains some free software. (One company founder said quite explicitly that they would put, into the free package they support, as little of their work as the community would stand for.)
Over the years, many companies have contributed to free software development. Some of these companies primarily developed non-free software, but the two activities were separate; thus, we could ignore their non-free products, and work with them on free software projects. Then we could honestly thank them afterward for their free software contributions, without talking about the rest of what they did.
We cannot do the same with these new companies, because they won't let us. These companies actively invite the public to lump all their activities together; they want us to regard their non-free software as favorably as we would regard a real contribution, although it is not one. They present themselves as “open source companies,” hoping that we will get a warm fuzzy feeling about them, and that we will be fuzzy-minded in applying it.
This manipulative practice would be no less harmful if it were done using the term “free software.” But companies do not seem to use the term “free software” that way; perhaps its association with idealism makes it seem unsuitable. The term “open source” opened the door for this.
At a trade show in late 1998, dedicated to the operating system often referred to as “Linux”, the featured speaker was an executive from a prominent software company. He was probably invited on account of his company's decision to “support” that system. Unfortunately, their form of “support” consists of releasing non-free software that works with the system—in other words, using our community as a market but not contributing to it.
He said, “There is no way we will make our product open source, but perhaps we will make it ‘internal’ open source. If we allow our customer support staff to have access to the source code, they could fix bugs for the customers, and we could provide a better product and better service.” (This is not an exact quote, as I did not write his words down, but it gets the gist.)
People in the audience afterward told me, “He just doesn't get the point.” But is that so? Which point did he not get?
He did not miss the point of the Open Source movement. That movement does not say users should have freedom, only that allowing more people to look at the source code and help improve it makes for faster and better development. The executive grasped that point completely; unwilling to carry out that approach in full, users included, he was considering implementing it partially, within the company.
The point that he missed is the point that “open source” was designed not to raise: the point that users deserve freedom.
Spreading the idea of freedom is a big job—it needs your help. That's why we stick to the term “free software” in the GNU Project, so we can help do that job. If you feel that freedom and community are important for their own sake—not just for the convenience they bring—please join us in using the term “free software”.
The free software movement was started by Richard M. Stallman and GNU in 1984, later the Free Software Foundation was founded.
Free software is defined by the offering of 4 basic freedoms:
• The freedom to run the program, for any purpose (freedom 0).
• The freedom to study how the program works, and adapt it to your needs (freedom 1). Access to the source code is a precondition for this.
• The freedom to redistribute copies so you can help your neighbor (freedom 2).
• The freedom to improve the program, and release your improvements to the public, so that the whole community benefits (freedom 3). Access to the source code is a precondition for this.
Non-free software is also called proprietary software. Free software should not be confused with freeware; freeware is free as in free beer, not as in freedom.
2 Benefits of Free and Open Source Software
These freedoms benefit users in many ways. Without access to the code and the right to modify it and distribute it, a distribution like openSUSE would not be possible at all.
1 Fix the software
These freedoms mean that you can fix bugs, which exist in all software, or you can change the software to do what you need it to do, or even fix security issues. In the case of proprietary software you can ask the provider to add functionality and fix bugs, and maybe they'll do it when it suits them, maybe not.
2 Share
Free software allows you to share software and thus help your friends and neighbours without you having to breach licenses.
3 Know and control what is going on
With proprietary software you can't know what a given program _really_ does. Some very well known proprietary software has been caught spying on users and sending information about their behaviour and such. Proprietary software also has a tendency to include various digital restrictions on what the user can do, when, for how long, etc. With free software you have access to the source code and can study what the program does and change it if you don't like it.
4 Technical benefits
Open source code makes it possible for more people to see the code and fix it, it can be developed faster and become better. This system of "peer review" can be compared to the way scientific research works. In comparison proprietary code is kept secret and rarely seen by anybody outside the company behind it.
5 Economic benefits
It's also a way in which companies can share development costs. For example Novell and Red Hat are competitors yet they develop many of the same programs and thus help each other. IBM and HP could also be seen as competitors yet they both contribute to the Linux kernel, etc., thus sharing development costs.
Free software makes a competitive market for support possible, potentially heightening the quality of support. With proprietary software only the provider who has access to the source code can realistically offer decent support, and thus has a kind of monopoly.
Where I can use Linux?
You can use Linux as Server Os or as stand alone Os on your PC. (But it is best suited for Server.) As a server Os it provides different services/network resources to client. Server Os must be:
• Stable
• Robust
• Secure
• High Performance
Linux offers all of the above characteristics plus its Open Source and Free OS. So Linux can be used as:
(1) On stand alone workstation/PC for word processing, graphics, software development, internet, e-mail, chatting, small personal database management system etc.
(2) In network environment as:
(A) File and Print or Application Server
Share the data, Connect the expensive device like printer and share it, e-mail within the LAN/intranet etc are some of the application.
[pic]
Linux Server with different Client Os
(B) Linux sever cab be connected to Internet, So that PC's on intranet can share the internet/e-mail etc. You can put your web sever that run your web site or transmit the information on the internet.
[pic]
Linux Server can act as Proxy/Mail/WWW/Router Server etc.
So you can use Linux for:
• Personal Work
• Web Server
• Software Development Workstation
• Workgroup Server
• In Data Center for various server activities such as FTP, Telnet, SSH, Web, Mail, Proxy, Proxy Cache Appliance etc
What Kernel Is?
Kernel is hart of Linux Os.
It manages resource of Linux Os. Resources means facilities available in Linux. For e.g. Facility to store data, print data on printer, memory, file management etc .
Kernel decides who will use this resource, for how long and when. It runs your programs (or set up to execute binary files).
The kernel acts as an intermediary between the computer hardware and various programs/application/shell.
[pic]
It's Memory resident portion of Linux. It performance following task :-
• I/O management
• Process management
• Device management
• File management
• Memory management
Unit 2
1. Linux Essential Commands
1 a. File Administration
|S.No |Commmands with Explanation |
|1. |ls [option(s)] [file(s)] If you run ls without any additional parameters, the program will list the contents of the current directory |
| |in short form. |
| |-l detailed list |
| |-a displays hidden files |
|2. |p [option(s)] sourcefile targetfile Copies sourcefile to targetfile. |
| |-i Waits for confirmation, if necessary, before an existing targetfile is overwritten |
| |-r Copies recursively (includes subdirectories) |
|3. |mv [option(s)] sourcefile targetfile Copies sourcefile to targetfile then deletes the original sourcefile. |
| |-b Creates a backup copy of the sourcefile before moving |
| |-i Waits for confirmation, if necessary, before an existing targetfile is overwritten |
|4. |rm [option(s)] file(s) Removes the specified files from the file system. Directories are not removed by rm unless the option -r is |
| |used. |
| |-r Deletes any existing subdirectories |
| |-i Waits for confirmation before deleting each file. |
|5. |ln [option(s)] sourcefile targetfile Creates an internal link from the sourcefile to the targetfile, under a different name. |
| |-s Creates a symbolic link |
|6. |cd [options(s)] [directory] Changes the current directory. |
| |cd without any parameters changes to the user's home directory. |
|7. |mkdir [option(s)] directoryname Creates a new directory. |
|8. |rmdir [option(s)] directoryname Deletes the specified directory, provided it is already empty. |
|9. |chown [option(s)] username.group file(s) Transfers the ownership of a file to the user with the specified user name. |
| |-R Changes files and directories in all subdirectories. |
|10. |chgrp [option(s)] groupname file(s) Transfers the group ownership of a given file to the group with the specified group name. The file |
| |owner can only change group ownership if a member of both the existing and the new group. |
|11. |chmod [options] mode file(s) Changes the access permissions.The mode parameter has three parts: group, access, and access type. group |
| |accepts the following characters: |
| |u user |
| |g group |
| |o others |
| |For access, access is granted by the + symbol and denied by the - symbol. |
| |The access type is controlled by the following options: |
| |r read |
| |w write |
| |x eXecute — executing files or changing to the directory. |
| |s Set uid bit — the application or program is started as if it were started by the owner of the file. |
|12. |gzip [parameters] file(s) This program compresses the contents of files, using complex mathematical algorithms. Files compressed in |
| |this way are given the extension .gz and need to be uncompressed before they can be used. To compress several files or even entire |
| |directories, use the tar command. |
| |-d decompresses the packed gzip files so they return to their original size and can be processed normally (like the command gunzip). |
|13. |tar options archive file(s) The tar puts one file or (usually) several files into an archive. Compression is optional. |
| |tar is a quite complex command with a number of options available. The most frequently used options are: |
| |-f Writes the output to a file and not to the screen as is usually the case |
| |-c Creates a new tar archive |
| |-r Adds files to an existing archive |
| |-t Outputs the contents of an archive |
| |-u Adds files, but only if they are newer than the files already contained in the archive |
| |-x Unpacks files from an archive (extraction) |
| |-z Packs the resulting archive with gzip |
| |-j Compresses the resulting archive with bzip2 |
| |-v Lists files processed |
| |The archive files created by tar end with .tar. If the tar archive was also compressed using gzip, the ending is .tgz or .tar.gz. If it|
| |was compressed using bzip2, .tar.bz2. |
|14. |locate pattern(s) The locate command can find in which directory a specified file is located. If desired, use wild cards to specify|
| |file names. The program is very speedy, as it uses a database specifically created for the purpose (rather than searching through the |
| |entire file system). The database can be generated by root with updatedb. |
|15. |find [option(s)] The find command allows you to search for a file in a given directory. The first argument |
| |specifies the directory in which to start the search. The option -name must be followed by a search string, which may also include wild|
| |cards. Unlike locate, which uses a database, find scans the actual directory. |
2 mands to Access File Contents
|S.No. |Commands with explanation |
| 1. |cat [option(s)] file(s) The cat command displays the contents of a file, printing the entire contents to the screen without |
| |interruption. |
| |-n Numbers the output on the left margin |
|2. |less [option(s)] file(s) This command can be used to browse the contents of the specified file. Scroll half a screen page up |
| |or down with PgUp and PgDn or a full screen page down with Space. Jump to the beginning or end of a file using Home and End. Press Q to|
| |exit the program. |
|3. |grep [option(s)] searchstring filenames |
| |The grep command finds a specific searchstring in the specified file(s). If the search string is found, the command displays the line |
| |in which the searchstring was found along with the file name. |
| |-i Ignores case |
| |-l Only displays the names of the respective files, but not the text lines |
| |-n Additionally displays the numbers of the lines in which it found a hit |
| |-l Only lists the files in which searchstring does not occur |
|4. |diff [option(s)] file1 file2 |
| |The diff command compares the contents of any two files. The output produced by the program lists all lines that do not match.This is |
| |frequently used by programmers who need only send their program alterations and not the entire source code. |
| |-q Only reports whether the two given files differ |
3 c. File Systems
1. mount [option(s)] [] mountpoint
This command can be used to mount any data media, such as hard disks, CD-ROM drives, and other drives, to a directory of the Linux file system.
-r mount read-only
-t filesystem
Specifies the file system. The most common are ext2 for Linux hard disks, msdos for MS-DOS media, vfat for the Windows file system, and iso9660 for CDs.
2. umount [option(s)] mountpoint
This command unmounts a mounted drive from the file system. To prevent data loss, run this command before taking a removable data medium from its drive. Normally, only root is allowed to run the commands mount and umount. To enable other users to run these commands, edit the /etc/fstab file to specify the option user for the respective drive.
2 d. System Commands
1 System Information
df [option(s)] [directory]
The df (disk free) command, when used without any options, displays information about the total disk space, the disk space currently in use, and the free space on all the mounted drives. If a directory is specified, the information is limited to the drive on which that directory is located.
-H : shows the number of occupied blocks in gigabytes, megabytes, or kilobytes — in human-readable format
-t : Type of file system (ext2, nfs, etc.)
du [option(s)] [path]
This command, when executed without any parameters, shows the total disk space occupied by files and subdirectories in the current directory.
-a : Displays the size of each individual file
-h : Output in human-readable form
-s : Displays only the calculated total size
free [option(s)]
The command free displays information about RAM and swap space usage, showing the total and the used amount in both categories.
-b : Output in bytes
-k : Output in kilobytes
-m : Output in megabytes
date [option(s)]
This simple program displays the current system time. If run as root, it can also be used to change the system time. Details about the program are available in date.
2 Processes
top [options(s)]
top provides a quick overview of the currently running processes. Press H to access a page that briefly explains the main options to customize the program.
ps [option(s)] [process ID]
If run without any options, this command displays a table of all your own programs or processes — those you started. The options for this command are not preceded by hyphen.
aux
Displays a detailed list of all processes, independent of the owner.
kill [option(s)] process ID
Unfortunately, sometimes a program cannot be terminated in the normal way. However, in most cases, you should still be able to stop such a runaway program by executing the kill command, specifying the respective process ID (see top and ps).
kill sends a TERM signal that instructs the program to shut itself down. If this does not help, the following parameter can be used:
-9
Sends a KILL signal instead of a TERM signal, with which the process really is annihilated by the operating system. This brings the specific processes to an end in almost all cases.
killall [option(s)] processname
This command is similar to kill, but uses the process name (instead of the process ID) as an argument, causing all processes with that name to be killed.
3 Network
ping [option(s)] host name|IP address
The ping command is the standard tool for testing the basic functionality of TCP/IP networks. It sends a small data packet to the destination host, requesting an immediate reply. If this works, ping displays a message to that effect, which indicates that the network link is basically functioning.
-c : number Determines the total number of packages to send and ends after they have been dispatched. By default, there is no limitation set.
-f flood ping: sends as many data packages as possible. A popular means, reserved to root, to test networks.
-i :value Specifies the interval between two data packages in seconds. Default: one second
nslookup
The Domain Name System resolves domain names to IP addresses. With this tool, send queries to information servers (DNS servers).
telnet [option(s)] host name or IP address
Telnet is actually an Internet protocol that enables you to work on remote hosts across a network. telnet is also the name of a Linux program that uses this protocol to enable operations on remote computers. Warning
4 Miscellaneous
passwd [option(s)] [username]
Users may change their own passwords at any time using this command. Furthermore, the administrator root can use the command to change the password of any user on the system.
su [option(s)] [username]
The su command makes it possible to log in under a different user name from a running session. When using the command without specifying a user name, you will be prompted for the root password. Specify a user name and the corresponding password to use the environment of the respective user. The password is not required from root, as root is authorized to assume the identity of any user.
halt [option(s)]
To avoid loss of data, you should always use this program to shut down your system.
reboot [option(s)]
Does the same as halt with the difference that the system performs an immediate reboot.
clear
This command cleans up the visible area of the console. It has no options.
2. File system in Linux
In a computer the hard disk forms a physical medium which can store files, and thus forms a file system. The major file system types in Linux are
1) EXT3
2) EXT4
3) VFAT
4) Swap
• Ext3 and Ext4 stands for extended file system. VFAT is a filesystem equivalent to the windows FAT ( File Allocation Table ) file system, it stands for Virtual FAT.
• Ext3 and Ext4 are used to create and access logical volume.
• VFAT is used in external medias like Pendrive and all.
• Swap is used to create a swap area in the hard disk, which can be used as a virtual memory. The total memory which a running application can see is the sum of physical memory ( RAM ) and the Virtual memory ( Swap ).
In Linux the file system is maintained in a hierachial method. The “/” forms the root of all file system, under which all other directories are mounted.
Mount points and their Usage
Directory Usage
• /bin:Binary directory; Stores commands used in Linux
• /boot:Store files like boot loader, required during boot time
• /dev:Device information directory. Device files are kept here
• /etc:System configuration files are stored here
• /home:Document directory of all normal users
• /root:Document directory of super-user
• /mnt:Mount directory for manual mounting
• /media:Auto-mount directory
• /lib:Shared libraries and kernel modules are stored here
• /lost + found:Back-up point for ext3 file system
• /proc:Process information directory. It provides interface to kernel data structures
• /tmp:Directory provided for storing temporary files
• /sbin:Directory for storing only default system commands
• /var:Varying file directory for storing regularly updating files
• /opt:Optional directory for installing additional software
Linux has three types of users
1) Super-user: Super-user in Linux is called “root”. Root user has complete previlage in Linux. Only he has the administratory power.
2) Normal-user: Normal user doesn't have administrator power. Normal users have only a limited access. It is the root user which creates the normal user. In certain Linux OS there is no root user, like in Ubuntu, Linuxmint etc. In such cases we can use commands to give normal user an administrative privilage.
3) System user :System users are the users created by applications in the system. For example in servers the application allows only authorised users to access its service.
• In Linux, everything is configured as a file. This includes not only text files, images and compiled programs (also referred to as executables), but also directories, partitions and hardware device drivers.
• Each filesystem (used in the first sense) contains a control block, which holds information about that filesystem. The other blocks in the filesystem are inodes, which contain information about individual files, and data blocks, which contain the information stored in the individual files.
o the Linux kernel, however, the filesystem is flat. That is, it does not (1) have a hierarchical structure, (2) differentiate between directories, files or programs or (3) identify files by names. Instead, the kernel uses inodes to represent each file.
• An inode is actually an entry in a list of inodes referred to as the inode list.
• Each inode contains information about a file including
(1) its inode number (a unique identification number)
(2) the owner and group associated with the file,
(3) the file type (for example, whether it is a regular file or a directory),
(4) the file's permission list,
(5) the file creation, access and modification times,
(6) the size of the file and
(7) the disk address (i.e., the location on the disk where the file is physically stored).
3.Vi Editor
Vi is a command line text editor. As you would be quite aware now, the command line is quite a different environment to your GUI. It's a single window with text input and output only. Vi has been designed to work within these limitations and many would argue, is actually quite powerful as a result. Vi is intended as a plain text editor (similar to Notepad on Windows, or Textedit on Mac) as opposed to a word processing suite such as Word or Pages. It does, however have a lot more power compared to Notepad or Textedit.
There are two modes in Vi. Insert (or Input) mode and Edit mode. In input mode you may input or enter content into the file. In edit mode you can move around the file, perform actions such as deleting, copying, search and replace, saving etc.
When we run vi we normally issue it with a single command line argument which is the file you would like to edit.
vi
• When you run this command it opens up the file.
• You always start off in edit mode so the first thing we are going to do is switch to insert mode by pressing i. You can tell when you are in insert mode as the bottom left corner will tell you.
• ZZ (Note: capitals) - Save and exit
• :q! - discard all changes, since the last save, and exit
• :w - save file but don't exit
• :wq - again, save and exit
Below are some of the many commands you may enter to move around the file. Have a play with them and see how they work.
• Arrow keys - move the cursor around
• j, k, h, l - move the cursor down, up, left and right (similar to the arrow keys)
• ^ (caret) - move cursor to beginning of current line
• $ - move cursor to end of the current line
• nG - move to the nth line (eg 5G moves to 5th line)
• G - move to the last line
• w - move to the beginning of the next word
• nw - move forward n word (eg 2w moves two words forwards)
• b - move to the beginning of the previous word
• nb - move back n word
• { - move backward one paragraph
• } - move forward one paragraph
Below are some of the many ways in which we may delete content within vi. Have a play with them now. (also check out the section below on undoing so that you can undo your deletes.)
• x - delete a single character
• nx - delete n characters (eg 5x deletes five characters)
• dd - delete the current line
• dn - d followed by a movement command. Delete to where the movement command would have taken you. (eg d5w means delete 5 words)
Undoing changes in vi is fairly easy. It is the character u.
• u - Undo the last action (you may keep pressing u to keep undoing)
• U (Note: capital) - Undo all changes to the current line
4.Linux Security Models
• SELinux is a system with MAC, or Mandatory Access Control. It implements a security policy called RBAC, Role-Based Access Control. This policy is implemented by DTAC, which is Dynamically Typed Access Control, which also translates as the domain name of Access Control System.
• SELinux is sponsored by the U.S. National Security Agency. The money goes to the team that is working on SELinux, the Secure Computing Corp. It should be noted that this company also owns all of the patents for the software.
• In 1992, a new idea for security resulted in a project called Distributed Trusted Match. The project developed some innovative solutions, which became part of an operating system called Fluke. Fluke evolved into Flux, which led to the development of Flask architecture. The Flask architecture was then integrated with the Linux kernel, and the whole newly created project was called SELinux.
• Linux was chosen because of two main features:
• growing popularity
• open developement environment
• Three elements of this system is very important.
1. The Kernel.: SELinux uses the Linux kernel called LSM, Linux Security Modules. This infrastructure provides all possible interfaces that allow you to fully control access to all system objects when they are initiated by user actions.
2.SELinux allows all programs a little bit of freedom
3.They define access rights, the right to pursue activities in the system, and the behavior of SELinux system.
• SELinux RBAC system with built-in rules that are implemented by DTAC. SELinux system also extends what is called the philosophy of SUID. Thus, it can also extend the understanding of the role of users in the system.
• SELinux is an implementation of a mandatory access control (MAC) security model in Linux operating system (OS). This mechanism resides inside the Linux kernel and checks for allowed operations after standard Linux discretionary access controls (DAC) are enabled
Information security is made up of the following main attributes:
•Availability - Prevention of loss of access to resources and data
•Integrity- Prevention of unauthorized modification of data
•Confidentiality- Prevention of unauthorized disclosure of data
Thesecurity kernel is made up of mechanisms that fall within the Trust Computing Base and implementsand enforces the reference monitor concept. The security kernel is made up of hardware,firmware, and software components that mediate all access and functions between subjects and objects. The security kernel is the core of the TCB and is the most commonlyused approach to building trusted computing systems. There are four main requirements of the security kernel:
•It must provide isolation for the processes carrying out the reference monitor
concept
•The reference monitor must be invoked for every access attempt and must be
impossible to circumvent. Thus, the reference monitor must be implemented
in a complete and foolproof way.
•The reference monitor must be verifiable as being correct. This means that all
decisions made by the reference monitor should be written to an audit log, andverified as being correct.
•It must be small enough to be able to be tested and verified in a complete and
comprehensive manner.
A security policy is a set of rules and practices dictating how sensitive information is managed, protected, and distributed. A security policy expresses exactly what the security level should be by setting the goals of what the security mechanisms are to accomplish.
Security policies that prevent information from flowing from a high security level to a lower security level are called multilevel security policies
. These types of policies permit a subject to access an object only if the subject’s security level is higher than or equal to the object’s classification.
A security model maps the abstract goals of the policy to information system terms by specifying explicit data structures and techniques necessary to enforce the security policy. A security model is usually represented in mathematics and analytical ideas, which are then mapped to system specifications, and then developed by programmers through programming code.
The security model takes this requirement and provides the necessary mathematical formulas, relationships, and structure to be followed to accomplish this goal.
Some security models enforce rules to protect confidentiality, such as the Bell- LaPadula model. Other models enforce rules to protect integrity, such as the Biba model. Formal security models, such as Bell-LaPadula and Biba, are used to provide high assurance in security. Informal models, such as Clark-Wilson, are used more as a framework to describe how security policies should be expressed and executed.
3 Understanding DAC and MAC Linux security models
As SELinux is based on the concept of MAC, it is very important to understand shortfalls of DAC (the default Linux security model) and advantages of MAC over DAC.
Under MAC, administrators control all interactions of software on the system. The concept of least privilege is used, and by default applications and users have no rights, as all rights must be granted by an administrator as part of the system’s security policy. Under DAC, files are owned by a user and that user has full control over them. An attacker who penetrates an account can do anything with the files owned by that user. For example, a hacker gaining access to a FTP server will have full control over all files owned by the FTP server account. Worse, if an application runs under the context of the root user (which is common for services like Web and FTP), an attacker will have full control over the entire system.
MAC provides each application with a virtual sandbox that only allows the application to perform the tasks it is designed for and are explicitly allowed in the security policy. For example, the Web server may only be able to read Web published files and serve them on a specified network port. An attacker penetrating it will not be able to perform any activities not expressly permitted by the security policy, even if the process is running as the root user.
Standard Unix permissions will still be present on the system, and will be consulted before the SELinux policy when access attempts are made. If the standard permissions would deny access, access is simply denied and SELinux is not involved. However if the standard file permissions would allow access, the SELinux policy is consulted and access is either allowed or denied based on the security contexts of the source process and the targeted object.
4 How SELinux defines subjects and objects
There are two important concepts, subjects and objects, in MAC's security context. A MAC (or non-discretionary access control) framework allows you to define permissions for how all processes, called subjects, interact with other parts of the system such as files, devices, sockets, ports, and other processes -- objects. This is done through an administratively-defined security policy over all processes and objects. These processes and objects are controlled through the kernel, and security decisions are made on all available information rather than just user identity. With this model, a process can be granted just the permissions it needs to be functional. This follows the principle of least privilege, which contrasts with the full privilege concept of DAC.
Under MAC, for example, users who have exposed their data using "chmod" are protected by the fact that their data is only associated with user home directories, and confined processes cannot touch those files without permission and purpose written into the policy.
5 SELinux security policies: Strict and targeted
SELinux follows the model of least-privilege. By default, everything is denied and then a policy is written that gives each element of the system (a service, program, user, process) only the access required to perform specified function. If a service, program or user tries to access or modify a file or resource that is not necessary for it to function then access is denied and the action is logged. Because SELinux is implemented within the kernel, individual applications do not need to be especially written or modified to work with SELinux. If SELinux blocks an action, this appears as just a normal "access denied" type error to the application.
The flow of SELinux with default targeted policy can be depicted as following logical block diagram:
Click on image for larger version
One of the most important concepts is SELinux policy. The model of least-privilege best describes the “strict” policy. SELinux allows different policies and the default policy in CentOS 5 and RHEL is the “targeted” policy that targets and confines key system processes. In RHEL, over 200 targets exist (including httpd, named, dhcpd, mysqld). Everything else on the system runs in an unconfined domain and is unaffected by SELinux. The goal is for every process that is installed and running at boot by default to be running in a confined domain. The targeted policy is designed to protect as many key processes as possible without adversely affecting the end user experience and most users should be totally unaware that SELinux is even running.
Another important concept is SELinux access control. There are three forms of access control, which are type enforcement (TE), role-based access control (RBAC) and multi-level security (MLS). Among these, TE is primary mechanism of access control in targeted policy.
6 Setting up the SELinux security context
It is very important to understand, that all processes and files in SELinux model posseses an SELinux security context.This security context can easily be displayed by using "-Z" flag as shown below:
[pic]
Click on image for larger version
Most of the SELinux troubleshooting revolves around security contexts of objects. These security contexts are in format of user:role:type:mls. Field "mls" is always hidden (as being default in targeted policy), so for example for hello.pl file, root is user, object_r is the role and type is httpd_sys_content_t. Within the default targeted policy, type is the important field used to implement TE, shown above.
Similarly you can list the security context of all processes running in Linux using same "-Z" flag with ps command, for example
#ps -efZ | grep mail
system_u:system_r:sendmail_t root 2661 1 0 12:30 ? 00:00:00 sendmail: accepting connections
system_u:system_r:sendmail_t smmsp 2670 1 0 12:30 ? 00:00:00 sendmail: Queue runner@01:00:00 for /var/spool/clientmqueue
The above output shows that Sendmail processes on our Linux server are running under "Sendmail_t" type domain.
2
3 5.Partitions creation:
4 5.1. Formating Partitions
At the shell prompt, I begin making the file systems on my partitions. Continuing with the example in this is:
|# mke2fs /dev/sda1 |
I need to do this for each of my partitions, but not for /dev/sda4 (my extended partition). Linux supports types of file systems other than ext2. You can find out what kinds your kernel supports by looking in: /usr/src/linux/include/linux/fs.h
The most common file systems can be made with programs in /sbin that start with "mk" like mkfs.msdos and mke2fs.
5 5.2. Activating Swap Space
To set up a swap partition:
|# mkswap -f /dev/hda5 |
To activate the swap area:
|# swapon /dev/hda5 |
Normally, the swap area is activated by the initialization scripts at boot time.
6 5.3. Mounting Partitions
Mounting a partition means attaching it to the linux file system. To mount a linux partition:
|# mount -t ext2 /dev/sda1 /opt |
-t ext2
File system type. Other types you are likely to use are:
• ext3 (journaling sile system based on ext2)
• msdos (DOS)
• hfs (mac)
• iso9660 (CDROM)
• nfs (network file system)
/dev/sda1
Device name. Other device names you are likely to use:
• /dev/hdb2 (second partition in second IDE drive)
• /dev/fd0 (floppy drive A)
• /dev/cdrom (CDROM)
/opt
mount point. This is where you want to "see" your partition. When you type ls /opt, you can see what is in /dev/sda1. If there are already some directories and/or files under /opt, they will be invisible after this mount command.
7 5.4. Some facts about file systems and fragmentation
Disk space is administered by the operating system in units of blocks and fragments of blocks. In ext2, fragments and blocks have to be of the same size, so we can limit our discussion to blocks.
Files come in any size. They don't end on block boundaries. So with every file a part of the last block of every file is wasted. Assuming that file sizes are random, there is approximately a half block of waste for each file on your disk. Tanenbaum calls this "internal fragmentation" in his book "Operating Systems".
You can guess the number of files on your disk by the number of allocated inodes on a disk. On my disk
|# df -i |
|Filesystem Inodes IUsed IFree %IUsed Mounted on |
|/dev/hda3 64256 12234 52022 19% / |
|/dev/hda5 96000 43058 52942 45% /var |
there are about 12000 files on / and about 44000 files on /var. At a block size of 1 KB, about 6+22 = 28 MB of disk space are lost in the tail blocks of files. Had I chosen a block size of 4 KB, I had lost 4 times this space.
Data transfer is faster for large contiguous chunks of data, though. That's why ext2 tries to preallocate space in units of 8 contigous blocks for growing files. Unused preallocation is released when the file is closed, so no space is wasted.
Noncontiguous placement of blocks in a file is bad for performance, since files are often accessed in a sequential manner. It forces the operating system to split a disk access and the disk to move the head. This is called "external fragmentation" or simply "fragmentation" and is a common problem with MS-DOS file systems. In conjunction with the abysmal buffer cache used by MS-DOS, the effects of file fragmentation on performance are very noticeable. DOS users are accustomed to defragging their disks every few weeks and some have even developed some ritualistic beliefs regarding defragmentation.
None of these habits should be carried over to Linux and ext2. Linux native file systems do not need defragmentation under normal use and this includes any condition with at least 5% of free space on a disk. There is a defragmentation tool for ext2 called defrag, but users are cautioned against casual use. A power outage during such an operation can trash your file system. Since you need to back up your data anyway, simply writing back from your copy will do the job.
The MS-DOS file system is also known to lose large amounts of disk space due to internal fragmentation. For partitions larger than 256 MB, DOS block sizes grow so large that they are no longer useful (This has been corrected to some extent with FAT32). Ext2 does not force you to choose large blocks for large file systems, except for very large file systems in the 0.5 TB range (that's terabytes with 1 TB equaling 1024 GB) and above, where small block sizes become inefficient. So unlike DOS there is no need to split up large disks into multiple partitions to keep block size down.
Use a 1Kb block size if you have many small files. For large partitions, 4Kb blocks are fine.
1 6.Shell Introduction
String operators in shell use unique among programming languages curly-bracket syntax. In shell any variable can be displayed as ${name_of_the_variable} instead of $name_of_the_variable. This notation was introduced to protect a variable name from merging with string that comes after it, but was extended to allow string operations on variables. Here is example in which it is used for separation of a variable $var and a string "_string" using curly brackets:
$ export var='test'
$ echo ${var}_string # var is a variable that uses syntax ${var} with the value test
$ echo $var_string # var_string is a variable that doesn't exist, echo doesn't print anything
In Korn 88 shell this notation was extended to allow expressions inside curvy brackets. For example
${var:=moo}
Each operation is encoded using special symbol or two symbols ( "digram", for example :- , := , etc). An argument that the operator may need is positioned after the symbol of the operation. Later this notation extended ksh93 and adopted by bash and other shells.
This "ksh-originated" group of operators is the most popular and probably the most widely used group of string-handling operators so it makes sense to learn them, if only in order to be able to modify old scripts.
6.1 Implementation of classic string operations in shell
Despite shell deficiencies in this area and idiosyncrasies preserved from 1970th most classic string operations can be implemented in shell. You can define functions that behave almost exactly like in Perl or other "more normal" language. In case shell facilities are not enough you can use AWK or Perl. It's actually sad that AWK was not integrated into shell.
6.1.1Length Operator
There are several ways to get length of the string.
• The simplest one is ${#varname}, which returns the length of the value of the variable as a character string. For example, if filename has the value ntp.conf, then ${#filename} would have the value 8.
• The second is to use built-in function expr, for example
expr length $string
or
expr "$string" : '.*'
• Additional example from Advanced Bash-Scripting Guide
stringZ=abcABC123ABCabc
echo ${#stringZ} # 15
echo `expr length $stringZ` # 15
echo `expr "$stringZ" : '.*'` # 15
• For checking if length of the string is zero you can use -n STRING operator.
More complex example. Here's the function for validating that that string is within a given max length. It requires two parameters, the actual string and the maximum length the string should be.
check_length()
# check_length
# to call: check_length string max_length_of_string
{
# check we have the right params
if (( $# != 2 )) ; then
echo "check_length need two parameters: a string and max_length"
return 1
fi
if (( ${#1} > $2 )) ; then
return 1
fi
return 0
}
You could call the function check_length like this:
#!/usr/bin/bash
# test_name
while :
do
echo -n "Enter customer name :"
read NAME
[ check_length $NAME 10 ] && break
echo "The string $NAME is longer then 10 characters"
done
echo $NAME
6.2Determining the Length of Matching Substring at Beginning of String
This is pretty rarely used capability of expr built-in but still sometimes it can be useful:
expr match "$string" '$substring'
where:
• String is any variable of literal string.
• $substring is a regular expression.
my_regex=abcABC123ABCabc
# |------|
echo `expr match "$my_regex" 'abc[A-Z]*.2'` # 8
echo `expr "$my_regex" : 'abc[A-Z]*.2'` # 8
7.String processing
7.1Manipulating Strings
Bash supports a surprising number of string manipulation operations. Unfortunately, these tools lack a unified focus. Some are a subset of parameter substitution, and others fall under the functionality of the UNIX expr command. This results in inconsistent command syntax and overlap of functionality, not to mention confusion.
String Length
${#string}
expr length $string
These are the equivalent of strlen() in C.
expr "$string" : '.*'
|stringZ=abcABC123ABCabc |
| |
|echo ${#stringZ} # 15 |
|echo `expr length $stringZ` # 15 |
|echo `expr "$stringZ" : '.*'` # 15 |
Example 7-1. Inserting a blank line between paragraphs in a text file
|#!/bin/bash |
|# paragraph-space.sh |
|# Ver. 2.1, Reldate 29Jul12 [fixup] |
| |
|# Inserts a blank line between paragraphs of a single-spaced text file. |
|# Usage: $0 read markings and then brows to the file and open it. synaptic will mark the softwares in the list automatically.
#3. Through terminal:
if you know the exact name of the software then you can install it through terminal by simply entering the command:
sudo apt-get update (to get the latest version)
sudo apt-get install software-package-name
that's it.....
if it says "unable to locate package..." then follow the instructions in the ppa installation section of this tutorial.
#4. PPA installation:
if your software's not in the software list then it may come from private package archives (PPA's).
these are private development of softwares so use it at your own risk.
steps:
1. search google for the ppa address for your software. (like ppa for package-name)
2. then add it to the repository by entering this command in terminal:
sudo add-apt-repository ppa:.....whatever_it_is
3. then run this command (must)
sudo apt-get update
4. now your desired software is in the list. so you can follow one of the above processes (#1,#2,#3)
Offline installation:
say, you downloaded your desired softwares from some website. in this case if you don't trust the origin of the software then don't install it or install it at your own risk.
Your downloaded softwares may come as a .zip, tar.gz, tar.bz2, .deb, .rpm, .tgz, tar.xz or any other types of archives.
if you are in linux mint or in ubuntu or in a debian based OS try to download .deb packages because it's easier to install in debian based OS.
#5. installing .deb packages:
through terminal:
cd path_to_the_directory_that_contains_the_.deb_file
sudo dpkg -i filename.deb
through gdebi package manager:
if gdebi is not installed then you have to install it through one of the processes #1,#2,#3 (requires internet connection)
1. then double click on the .deb file or open the file with gdebi package manager and click install.
2. it will be istalled soon.
#6. installing .rpm packages:
rpm has to be installed in the system, otherwise follow one of the processes #1,#2,#3 to install rpm (requires internet connection)
code:
cd path_to_the_directory_that_contains_the_.rpm_file
sudo rpm -i filename.rpm
#7. installing from archives(.zip tar.gz.......etc):
these archives generally conatains the source of the package. each of them generally has a different approach to install. I will be discussing a common method which will supposedly work for all of them.
general requirements:
1. flex
2. bison or bison++
3. python
As these archives contains the source, your system needs the required programming languages to compile and build the source. so the general requirement packages stated above may not be sufficient for you. in that case you have to install the required packages through one of the processes #1,#2,#3 (requires internet connection). you can know about the dependencies about your software in a readme file included in the archives.
steps:
1. open the archives with archive manager by double clicking it, then extract it.
2. code:
cd path-to-the-extracted-folder
3. inside the extracted folder look carefully....
a. if you find a file named configure then :
code:
./configure
make
sudo make install
if the first code fails to execute then run this code before above codes:
chmod +x configure
b. if you find a file named install.sh then
code :
chmod +x install.sh
./install.sh or sudo ./install.sh (if it needs root permission)
or you can double click it and select run in terminal or simply run
N.B : sometimes there is a file, something like your_software_name.sh is found instead of install.sh. for this case you have to replace install.sh with the correct name in the previous codes.
c. if you find a file named install then
code:
chmod +x install
./install or sudo ./install (if it needs root permission)
or you can double click it and select run in terminal or simply run
d. if you find a file named make (if there is no configure file) then
code:
make
sudo make install
e. If you still can't find the required files
then it may be in a special folder (generally in a folder named 'bin'). move to this folder with cd command with appropriate path and then look again and follow the same process.
#8. pre installed archives:
some packages are archived as pre installed packages i.e you don't have to install them , you just need to extract them in a secure place and theres an executable file (name is generally same as the software name) in the extracted folder or in child folders like bin,build etc. you have to find it and make it executable.
Example: eclipse, adt bundle (android developing tool)
code to make executable:
chmod +x filename_with_exact_path
then you can run it with :
code:
filename
or double clicking it and selecting run in terminal or run, whatever supports your software.
#9. installing .sh files:
some softwares come with a .sh file to install it
chmod +x filename.sh
./filename.sh or sudo ./filename.sh (if it needs root permission)
or double click it and select run in terminal or run, whatever supports your software.
#10. installing .run files:
some softwares come with a .run file to install it
chmod +x filename.run
./filename.run or sudo ./filename.run (if it needs root permission)
or double click it and select run in terminal or run, whatever supports your software.
Additional info about offline installation:
Often, softwares generally have a lot of dependencies. You need to download all of them, which sometimes can be very difficult and tiring. for this you can follow the instructions here which will save your time and effort.
UNIT-3
Installation and setup of the Apache Web Server (Windows NT)
Apache is the Web Server most frequently used on the Internet today. This tutorial covers its installation and and configuration in Windows NT. You should download Apache for free and then install and configure it. There is another tutorial for Apache in Linux.
• Installing Apache
Go to the Apache Win32 download directory. You will need to download two pieces of software in a temporary directory. The first is the new Microsoft MSI package installer (MSIEXEC) instmsi.exe, standard in Windows Me and 2000, recently made available by Microsoft for Windows NT and 95/98. The second is the file apache 1.3.x.x-win32-xxxx.msi file with the apache server packaged to be installed with MSIEXEC. Run the instmsi.exe file to install MSIEXEC. Once MSIEXEC is installed then double-click on the 1.3.x.x-win32-xxxx.msi file. Install apache in C:\Apache instead of the default in Program Files. During the installation you will be asked for a server name (you can enter localhost, or an IP number, or yet a FQDN), and a domain name -- use the domain of your provider. Note: be sure you have Perl downloaded and installed before you install Apache, if you intend to use Perl to write CGI scripts.
The package will automatically create all the directories for you besides installing the software. The document root will be in c:/Apache/Apache/htdocs. The server root will be located in c:/Apache/Apache. The path to the apache program is C:/Apache/Apache/Apache.exe, but in NT it runs as a service, which is also automatically installed.
In addition a variety of directories are created: cgi-bin, htdocs, icons, include, lib, libexec, etc. In the htdocs directory you will store the Web pages for your server. In the cgi-bin directory you will store your CGI programs.
In the server root three directories are created: conf, logs and modules. In the conf directory you will find the three basic Apache configuration files: httpd.conf, access.conf and srm.conf. In the logs directory you will find the access and error logs. The modules directory is where the Apache modules are resident.
• Configuring Apache in NT
You need to configure (change) the C:/Apache/Apache/conf/httpd.conf file. The other two configuration files are considered obsolete and should not be changed.
o httpd.conf: this is the overall configuration file. Open the file httpd.conf in Notepad or any other text editor like PFE.
Find first the line #BindAddress * and delete the # (uncomment it) to make it active. Find the line ServerAdmin and enter your e-mail address, and look for the line ServerName and enter the FQDN, or IP number of your machine, or yet localhost. The server comes configured to run in standalone mode, to listen in port 80, and you don't need to change these options. If you installed Apache in C:/Apache the document root directory is written by default in the line
DocumentRoot "c:/Apache/Apache/htdocs"
Look for the section that starts with and look for the Options line and change it to allow Server Side Includes, but disabling scripts to be run from a Web page, as follows: Options Indexes Includes FollowSymLinksIncludesNOEXEC. This will allow the dynamic dating of your changes, the dynamic display of time and date in your pages, but will prevent scripts to be run outside of your cgi-bin directory. For example, the HTML code
will be displayed as Friday January 18, 2019 -- 04:42 AM
You may want to change the order of your index files, making index.shtml (the SSI version) to be the first to be displayed by the browsers. In order to do so, look for the line DirectoryIndex and change it to DirectoryIndex index.shtml index.html index.htm
Look for the line #ScriptInterpreterSource registry and remove the # (uncomment it). This will allow scripts written in different languages to use their extention associations in Windows. For example, hello.pl would be associated with Perl. In case you wish to keep it commented -- do not use associations, then the first line of a script would be like in Linux/Unix -- a shebang line as follows:
#!c:/Perl/bin/Perl.exe
Check that your cgi-bin directory is properly identified as follows: ScriptAlias /cgi-bin/ "C:/Apache/Apache/cgi-bin/" . If you moved your document root path change this line accordingly.
Look for the line starting with # AddHandler. If you want your CGI script files to be identified with .cgi be sure that the line AddHandlercgi-script .cgi is not commented out (marked with in # in front). To complete the installation of server side includes be sure that the following two lines are present and uncommented (without the # in front):
AddType text/html .shtml
AddHandler server-parsed .shtml
Finally, to make your Web server support image maps be sure that the following line exists: AddHandler imap-file map
• Starting Apache
Apache runs as a service in Windows NT, therefore you can start, stop and restart the Apache Web server using the Services applet in the Windows NT Control Panel.
How to Install Nginx on Ubuntu 16.04
Nginx is one of the most popular open source web server software. It is widely used for load balancing, reverse proxying, media streaming and much more. Nginx is the only all-in-one high-performance HTTP server, designed to serve a high volume of concurrent connections, as an alternative to Apache web server. Today Nginx is installed on more than 450 million servers worldwide and is used by one of the highest traffic websites on the web like Dropbox, Zynga games, Netflix. In this tutorial, we will show you how to Install Nginx on Ubuntu 16.04.
[pic]There is no question that Nginx is the best-in-class web server and application delivery solution. Installing Nginx on Ubuntu 16.04 is an easy task, just carefully follow the steps below and you should have Nginx installed on your Ubuntu server in less than 10 min. This guide should work on other Ubuntu-based Linux VPS systems as well but was tested and written for Ubuntu 16.04 VPS. Let’s get started.
1 Step 1: Install Nginx on Your Ubuntu Server
Nginx is available in the Ubuntu package repositories simple. First, update the apt cache with the following command:
sudo apt update
and install Nginx by issuing:
sudo apt install nginx
Once the installation is completed Nginx will be automatically started.
You can make sure that Nginx service is running with the following command:
sudosystemctl status nginx
The output should look like below:
● nginx.service - A high performance web server and a reverse proxy server
Loaded: loaded (/lib/systemd/system/nginx.service; enabled; vendor preset: enabled)
Active: active (running) since Sat 2018-03-31 01:50:44 CDT; 8s ago
Main PID: 716 (nginx)
CGroup: /system.slice/nginx.service
├─716 nginx: master process /usr/sbin/nginx -g daemon on; master_process on
├─717 nginx: worker process
├─718 nginx: worker process
├─719 nginx: worker process
└─720 nginx: worker process
2 Step 2: Open Firewall Ports
If you are using ufw you need to open HTTP port 80 and/or HTTPS port 433. Ufw comes with profiles based on the default ports of most common daemons and programs.
To open both Nginx ports run the following command:
sudoufw allow 'Nginx Full'
To verify the change run:
sudoufw status
The output should look like below:
Status: active
To Action From
-- ------ ----
Nginx Full ALLOW Anywhere
Nginx Full (v6) ALLOW Anywhere (v6)
You can now open your browser, enter your server IP address into your browser address bar and you should see the default Nginx page.
[pic]
3 Step 3: Managing Nginx Service
You can manage the Nginx service same as any other systemd unit.
Start the nginx service with the following command:
sudosystemctl start nginx
Stop the service with:
sudosystemctl stop nginx
Restart the service with:
sudosystemctl restart nginx
Check the status of the service with:
sudosystemctl status nginx
Enable the service on system boot with:
sudosystemctl enable nginx
Disable the service on system boot with:
sudosystemctl disable nginx
4 Step 4: Create a New Server Block
The default Nginx installation will have one server block enabled with a document root set to /var/www/html.
In this guide, we will create a new server block for the domain and set the document root to /var/www/.
First, create the domain document root with the following command:
sudomkdir -p /var/www/
and then create an index.html file with the following content:
sudo vim /var/www/index.html
server block
Next, create a new server block with the following content:
sudo vim /etc/nginx/sites-available/.conf
server {
listen 80;
listen [::]:80;
server_name ;
root /var/www/;
index index.html;
location / {
try_files $uri $uri/ =404;
}
}
Activate the server block by creating a symbolic link :
sudo ln -s /etc/nginx/sites-available/.conf /etc/nginx/sites-enabled/.conf
5 Step 5: Restart Nginx
Test the Nginx configuration and restart nginx:
sudonginx -t
sudosystemctl restart nginx
Now if you enter into your browser address bar you should see server block.
[pic]
[pic]Of course, you don’t have to install Nginx on Ubuntu 16.04, if you use one of our Optimized Nginx Web Hosting Solutions, in which case you can simply ask our expert Linux admins to install and configure Nginx on Ubuntu 16.04 for you. They are available 24×7 and will take care of your request immediately.
Eclipse Tutorial
In the context of computing, Eclipse is an integrated development environment (IDE) for developing applications using the Java programming language and other programming languages such as C/C++, Python, PERL, Ruby etc.
The Eclipse platform which provides the foundation for the Eclipse IDE is composed of plug-ins and is designed to be extensible using additional plug-ins. Developed using Java, the Eclipse platform can be used to develop rich client applications, integrated development environments and other tools. Eclipse can be used as an IDE for any programming language for which a plug-in is available.
The Java Development Tools (JDT) project provides a plug-in that allows Eclipse to be used as a Java IDE, PyDev is a plugin that allows Eclipse to be used as a Python IDE, C/C++ Development Tools (CDT) is a plug-in that allows Eclipse to be used for developing application using C/C++, the Eclipse Scala plug-in allows Eclipse to be used an IDE to develop Scala applications and PHPeclipse is a plug-in to eclipse that provides complete development tool for PHP.
1 Downloading Eclipse
You can download eclipse from . The download page lists a number of flavors of eclipse.
[pic]
The capabilities of each packaging of eclipse are different. Java developers typically use Eclipse Classic or Eclipse IDE for developing Java applications.
The drop down box in the right corner of the download page allows you to set the operating system on which eclipse is to be installed. You can choose between Windows, Linux and Mac. Eclipse is packaged as a zip file.
2 Installing Eclipse
To install on windows, you need a tool that can extract the contents of a zip file. For example you can use −
• 7-zip
• PeaZip
• IZArc
Using any one of these tools, extract the contents of the eclipse zip file to any folder of your choice.
[pic]
3 Launching Eclipse
On the windows platform, if you extracted the contents of the zip file to c:\, then you can start eclipse by using c:\eclipse\eclipse.exe
When eclipse starts up for the first time it prompts you for the location of the workspace folder. All your data will be stored in the workspace folder. You can accept the default or choose a new location.
[pic]
If you select "Use this as the default and do not ask again", this dialog box will not come up again. You can change this preference using the Workspaces Preference Page. See the Preference tutorial page for more details.
Cloud Computing with Open Source Tool :OpenStack
OpenStack is a especially scalable open source cloud operating system that is a global alliance of developers and cloud computing technologists producing the ubiquitous open source cloud computing platform for public and private clouds. OpenStack provides series of interrelated projects delivering various components for a cloud infrastructure solution as well as controls large pools of storage, compute and networking resources throughout a datacenter that all managed through a Dashboard(Horizon) that gives administrators control while empowering their users to provision resources through a web interface.In this paper, we present a overview of Cloud Computing Platform such as, Openstack, Eucalyptus ,CloudStack and Opennebula which is open source software, cloud computing layered model, components of OpenStack, architecture of OpenStack.
OpenStack Platform IaaS cloud is implemented as a collection of interacting services
that control compute, storage, and networking resources. The cloud can be managed with a web-
based dashboard or command-line clients, which allow administrators to control, provision, and
automate OpenStack resources. OpenStack also has an extensive API, which is also available to all
cloud users.
OpenStack Networking handles creation and management of a virtual networking infrastructure in
the OpenStack cloud. Infrastructure elements include networks, subnets, and routers. You can also
deploy advanced services such as firewalls or virtual private networks (VPN).
OpenStack Networking provides cloud administrators with flexibility to decide which individual
services to run on which physical systems. All service daemons can be run on a single physical host
for evaluation purposes. Alternatively, each service can have a unique physical host or replicated
across multiple hosts to provide redundancy.
Because OpenStack Networking is software-defined, it can react in real-time to changing network
needs, such as creation and assignment of new IP addresses.
OpenStack Networking advantages include:
Users can create networks, control traffic, and connect servers and devices to one or more
networks.
Flexible networking models can adapt to the network volume and tenancy.
IP addresses can be dedicated or floating, where floating IPs can be used for dynamic traffic
rerouting.
If using VLAN networking, you can use a maximum of 4094 VLANs (4094 networks), where 4094
= 2^12 (minus 2 unusable) network addresses, which is imposed by the 12-bit header limitation.
If using VXLAN tunnel-based networks, the VNI (Virtual Network Identifier) can use a 24-bit
header, which will essentially allow around 16 million unique addresses/networks.
1 What is Version Control System?
Version Control System (VCS) is a software that helps software developers to work together and maintain a complete history of their work.
Following are the goals of a Version Control System.
• Allow developers to work simultaneously.
• Do not overwrite each other’s changes.
• Maintain history of every version of everything.
A VCS is divided into two categories.
• Centralized Version Control System (CVCS), and
• Distributed/Decentralized Version Control System (DVCS).
In this tutorial, we will concentrate only on the Centralized Version Control System and especially Subversion. Subversion falls under centralized version control system, meaning that it uses central server to store all files and enables team collaboration.
2 Version Control Terminologies
Let us start by discussing some of the terms that we will be using in this tutorial.
• Repository: A repository is the heart of any version control system. It is the central place where developers store all their work. Repository not only stores files but also the history. Repository is accessed over a network, acting as a server and version control tool acting as a client. Clients can connect to the repository, and then they can store/retrieve their changes to/from repository. By storing changes, a client makes these changes available to other people and by retrieving changes, a client takes other people's changes as a working copy.
• Trunk: The trunk is a directory where all the main development happens and is usually checked out by developers to work on the project.
• Tags : The tags directory is used to store named snapshots of the project. Tag operation allows to give descriptive and memorable names to specific version in the repository.
For example, LAST_STABLE_CODE_BEFORE_EMAIL_SUPPORT is more memorable than
Repository UUID: 7ceef8cb-3799-40dd-a067-c216ec2e5247 and
Revision: 13
• Branches: Branch operation is used to create another line of development. It is useful when you want your development process to fork off into two different directions. For example, when you release version 5.0, you might want to create a branch so that development of 6.0 features can be kept separate from 5.0 bug-fixes.
• Working copy: Working copy is a snapshot of the repository. The repository is shared by all the teams, but people do not modify it directly. Instead each developer checks out the working copy. The working copy is a private workplace where developers can do their work remaining isolated from the rest of the team.
• Commit changes: Commit is a process of storing changes from private workplace to central server. After commit, changes are made available to all the team. Other developers can retrieve these changes by updating their working copy. Commit is an atomic operation. Either the whole commit succeeds or is rolled back. Users never see half finished commit.
Git Tutorial
Git is a distributed revision control and source code management system with an emphasis on speed. Git was initially designed and developed by Linus Torvalds for Linux kernel development. Git is a free software distributed under the terms of the GNU General Public License version 2.
This tutorial explains how to use Git for project version control in a distributed environment while working on web-based and non web-based applications development.
Audience
This tutorial will help beginners learn the basic functionality of Git version control system. After completing this tutorial, you will find yourself at a moderate level of expertise in using Git version control system from where you can take yourself to the next levels.
Prerequisites
We assume that you are going to use Git to handle all levels of Java and Non-Java projects. So it will be good if you have some amount of exposure to software development life cycle and working knowledge of developing web-based and non web-based applications.
Git - Basic Concepts
Version Control System (VCS) is a software that helps software developers to work together and maintain a complete history of their work.
Listed below are the functions of a VCS:
• Allows developers to work simultaneously.
• Does not allow overwriting each other’s changes.
• Maintains a history of every version.
Following are the types of VCS:
• Centralized version control system (CVCS).
• Distributed/Decentralized version control system (DVCS).
In this chapter, we will concentrate only on distributed version control system and especially on Git. Git falls under distributed version control system.
1 Distributed Version Control System
Centralized version control system (CVCS) uses a central server to store all files and enables team collaboration. But the major drawback of CVCS is its single point of failure, i.e., failure of the central server. Unfortunately, if the central server goes down for an hour, then during that hour, no one can collaborate at all. And even in a worst case, if the disk of the central server gets corrupted and proper backup has not been taken, then you will lose the entire history of the project. Here, distributed version control system (DVCS) comes into picture.
DVCS clients not only check out the latest snapshot of the directory but they also fully mirror the repository. If the server goes down, then the repository from any client can be copied back to the server to restore it. Every checkout is a full backup of the repository. Git does not rely on the central server and that is why you can perform many operations when you are offline. You can commit changes, create branches, view logs, and perform other operations when you are offline. You require network connection only to publish your changes and take the latest changes.
2 Advantages of Git
1 Free and open source
Git is released under GPL’s open source license. It is available freely over the internet. You can use Git to manage property projects without paying a single penny. As it is an open source, you can download its source code and also perform changes according to your requirements.
2 Fast and small
As most of the operations are performed locally, it gives a huge benefit in terms of speed. Git does not rely on the central server; that is why, there is no need to interact with the remote server for every operation. The core part of Git is written in C, which avoids runtime overheads associated with other high-level languages. Though Git mirrors entire repository, the size of the data on the client side is small. This illustrates the efficiency of Git at compressing and storing data on the client side.
3 Implicit backup
The chances of losing data are very rare when there are multiple copies of it. Data present on any client side mirrors the repository, hence it can be used in the event of a crash or disk corruption.
4 Security
Git uses a common cryptographic hash function called secure hash function (SHA1), to name and identify objects within its database. Every file and commit is check-summed and retrieved by its checksum at the time of checkout. It implies that, it is impossible to change file, date, and commit message and any other data from the Git database without knowing Git.
5 No need of powerful hardware
In case of CVCS, the central server needs to be powerful enough to serve requests of the entire team. For smaller teams, it is not an issue, but as the team size grows, the hardware limitations of the server can be a performance bottleneck. In case of DVCS, developers don’t interact with the server unless they need to push or pull changes. All the heavy lifting happens on the client side, so the server hardware can be very simple indeed.
6 Easier branching
CVCS uses cheap copy mechanism, If we create a new branch, it will copy all the codes to the new branch, so it is time-consuming and not efficient. Also, deletion and merging of branches in CVCS is complicated and time-consuming. But branch management with Git is very simple. It takes only a few seconds to create, delete, and merge branches.
CVS Tutorial (Concurrent Versions System)
1 Basic description of the CVS system.
CVS is a system that lets groups of people work simultaneously on groups of files (for instance program sources).
It works by holding a central `repository' of the most recent version of the files. You may at any time create a personal copy of these files by `checking out' the files from the repository into one of your directories. If at a later date newer versions of the files are put in the repository, you can `update' your copy.
You may edit your copy of the files freely. If new versions of the files have been put in the repository in the meantime, doing an update merges the changes in the central copy into your copy.
When you are satisfied with the changes you have made in your copy of the files, you can `commit' them into the central repository.
When you are finally done with your personal copy of the files, you can `release' them and then remove them.
2 Some basic words and descriptions.
Repository
The directory storing the master copies of the files. The main or master repository is a tree of directories.
Module
A specific directory (or mini-tree of directories) in the main repository. Modules are defined in the CVS modules file.
RCS
Revision Control System. A lower-level set of utilities on which CVS is layered.
Check out
To make a copy of a file from its repository that can be worked on or examined.
Revision
A numerical or alpha-numerical tag identifying the version of a file.
3 Look at the CVS basic command set.
Most of the below commands should be executing while in the directory you checked out. If you did a cvs checkout malloc then you should be in the malloc sub-directory to execute most of these commands. cvs release is different and must be executed from the directory above.
cvs checkout (or cvs co)
To make a local copy of a module's files from the repository execute cvs checkout module where module is an entry in your modules file (see below). This will create a sub-directory module and check-out the files from the repository into the sub-directory for you to work on.
cvs update
To update your copy of a module with any changes from the central repository, execute cvs update. This will tell you which files have been updated (their names are displayed with a U before them), and which have been modified by you and not yet committed (preceded by an M). It can be that when you do an update, the changes in the central copy clash with changes you have made in your own copy. You will be warned of any files that contain clashes by a preceding C. Inside the files the clashes will be marked in the file surrounded by lines of the form >. You have to resolve the clashes in your copy by hand. After an update where there have been clashes, your original version of the file is saved as `.#file.version'. If you feel you have messed up a file and wish to have CVS forget about your changes and go back to the version from the repository, delete the file and do an cvs update. CVS will announce that the file has been "lost" and will give you a fresh copy.
cvs commit
When you think your files are ready to be merged back into the repository for the rest of your developers to see, execute cvs commit. You will be put in an editor to make a message that describes the changes that you have made (for future reference). Your changes will then be added to the central copy. When you do a commit, if you haven't updated to the most recent version of the files, CVS tells you this; then you have to first update, resolve any possible clashes, and then redo the commit.
cvs add and cvs remove
It can be that the changes you want to make involve a completely new file, or removing an existing one. The commands to use here are:
cvs add `filename'
cvs remove `filename'
You still have to do a commit after these commands to make the additions and removes actually take affect. You may make any number of new files in your copy of the repository, but they will not be committed to the central copy unless you do a cvs add. CVS remove does not actually remove the files from the repository. It only removes them from the "current list" and puts the files in the CVS Attic. When another person checks out the module in the future they will not get the files that were removed. But if you ask for older versions that had the file before it was removed, the file will be checked out of the Attic.
cvs release
When you are done with your local copy of the files for the time being and want to remove your local copy use cvs release module. This must be done in the directory above the module sub-directory you which to release. It safely cancels the effects of cvs checkout. Usually you should do a commit first. If you wish to have CVS also remove the module sub-directory and your local copy of the files then your cvs release -d module. NOTE: Take your time here. CVS will inform you of files that may have changed or it does not know about (watch for the ? lines) and then with ask you to confirm this action. Make sure you want to do this.
cvs log
To see the commit messages for files, and who made them, use:
cvs log [`filename(s)']
cvs diff
To see the differences between your version of the files and the version in the repository do:
cvs diff [`filename(s)']
cvs tag
One of the exciting features of CVS is its ability to mark all the files in a module at once with a symbolic name. You can say `this copy of my files is version 3'. And then later say `this file I am working on looked better in version 3 so check out the copy that I marked as version 3.' Use cvs tag to tag the version of the files that you have checked out. You can then at a later date retrieve this version of the files with the tag.
cvs tag tag-name [filenames]
Later you can do:
cvs co -r tag-name module
cvsrtag
Like tag, rtag marks the current versions of files but it does not work on your local copies but on the files in the repository. To tag all my libraries with a version name I can do:
cvsrtag LIBRARY_2_0 lib
This will recursively go through all the repository directories below lib and add the LIBRARY_2_0 tag to each file. This is one of the most useful features of CVS (IMHO). Use this feature if you about to release a copy of the files to the outside world or just want to mark a point in the developmental progression of the files.
cvs history
To find out information about your CVS repositories use the cvs history command. By default history will show you all the entries that correspond to you. Use the -a option to show information about everyone.
cvs history -a -o shows you (a)ll the checked (o)ut modules
cvs history -a -T reports (a)ll the r(T)ags for the modules
cvs history -a -e reports (a)ll the information about (e)verything
GitHub – Open source Version Control System
Collaboration between software developers enhances their productivity which helps them collate new ideas. Developers with relevant skills can contribute to a project to add more features to it or even fix the bugs. A software goes through many changes in its development lifecycle and developers constantly make modifications to the program code, release new versions and patche/fix functionality issues. It is important to be updated with these variations because it helps in effective coordination among the developers in building new software as well as managing them. Sharing Platforms like GitHub serves this purpose.
What is GitHub?
It is an open source Version Control System initiated by Linus Torvalds. Generally, When developers develop any applications they often make revisions or modifications to it for improvement. A version control system is a tool that maintains these changes in a central repository which offers Source Code Management (SCM). Other developers can see these changes, download them and contribute by developing and uploading a new code in the central repository. Git is a command line tool whereas the central repository is a web-based GUI and is called GitHub().
Note: Subversion, CVS, Mercurial are also version control tools but not popular as Git.
What are the features of GitHub?
GitHub is more popular than any other version control systems because it stores file changes more efficiently and ensures file integrity. You can see the history of a particular project. Following are the features of GitHub:
Repository
A repository or repo is a collection of all the files (images, videos, spreadsheets, and datasets etc.) of a project. Each project has a repository which can be accessed through a URL. You can create a repository in your GitHub account. Also note that the public repositories are free of cost while the private repositories are not.
Social Networking
GitHub offers a social networking to like-minded users. Each member has their own profile. Members can follow other members and they can take part in discussions on projects and seek the expertise of other developers.
How to create and manage SourceForge
SourceForge
is a free and well-established open-source project repository. It currently hosts over 430,000 open-source projects and is used by over 3.7 million registered users
SourceForge provides a project with myriad tools for open-source development and community engagement.
• Source code repository
SourceForge allows you to store your project's source code under revision control. SourceForge can provide you with a Git, Mercurial, or Subversion repository.
• File space
This is used to store project-related materials such as source and binary release bundles, documentation not suited for a wiki (e.g. user or API documentation in HTML form) and any files you don't want to hold under revision control.
• Web pages
You can set up web pages to describe your project - SourceForge can host your web site for you.
• Wiki
Wikis can be used for documentation. They often have a role similar to web pages, but are quicker and easier to put together and are useful recording information about a project that you edit more frequently than web pages e.g. FAQs, how-tos and information for users and developers.
• Forums
Forums provide a publicly available area where you and your users can discuss your project.
• Email lists
Mailing lists are generally used for project announcements and discussions.
• Blog
Blogs provide an informal route for interacting with your community. It's your chance to describe what's going on in your project, the problems that crop up, what you had for breakfast - pretty much anything.
• MySQL database
For use by resources hosted within your project file space.
• Statistics
Statistics allow you to monitor the uptake of your project, by recording metrics such as downloads and the number of commits to your source code.
• Ticketing system
A ticketing system is used to register issues and assign effort to understand or resolve them. They're generally used for bug tracking or dealing with support queries.
1 Adding the Repository
To configure a SourceForge repository, first proceed to add the repository and select SourceForge from the Hosting type field.
1 Step 1: Link Your SourceForge Account
You will need to link an account on SourceForge to Review Board, so that Review Board can access content from the repository. If you’ve already linked an account with sufficient access to the repository, you can use that instead.
If you’re linking for the first time, you’ll need to make sure you have your username and password handy.
Fill out the following fields:
Account username:
The username used to log into your SourceForge account. This is not your e-mail address.
Account password:
The password used to log into your SourceForge account.
Due to the way Review Board interacts with your repositories, your password will be stored in encrypted form in the database.
The account will be linked when the repository is saved. If there are errors authenticating the user or retrieving an access token, you will be prompted to fix them.
2 Step 2: Provide Repository Information
Next, you’ll need to fill out the following fields:
Repository type:
The type of repository you’re adding. This can be “Bazaar”, “CVS”, “Mercurial”, or “Subversion”.
Project name:
The identifier for your SourceForge project.
This is the name shown in the URL for your project page. It would be the project_name in .
3 Step 3: Choose a Bug Tracker
If you’re using the issue tracking feature on this repository, you can simply check the Use hosting service’s bug tracker checkbox. All bug IDs will link to the appropriate issues for your repository.
If you’re using a separate bug tracker, or a separate project on SourceForge, you can leave the checkbox unchecked and choose a bug tracker from the list.
Learn more about bug tracker configuration.
4 Step 4: Manage Access Control
You can now choose who should have access to this repository (both posting against it and viewing review requests posted against it). This is separate from SourceForge’s own access controls.
Learn more about access control.
5 Step 5: Save the Repository
At this point, you should be able to save the repository. If saving succeeds, you’re done! You can start posting changes for review.
Overview of Google Apps Script
Google Apps Script is a rapid application development platform that makes it fast and easy to create business applications that integrate with G Suite. You write code in JavaScript and have access to built-in libraries for favorite G Suite applications like Gmail, Calendar, Drive, and more. There's nothing to install—we give you a code editor right in your browser, and your scripts run on Google's servers.
If you're new to JavaScript, Codecademy offers a number of JavaScript courses. (Note that these courses were not developed by and are not associated with Google.)
1 What can Apps Script do?
Apps Script is versatile. Among other things, you can:
• Add custom menus, dialogs, and sidebars to Google Docs, Sheets, and Forms.
• Write custom functions and macros for Google Sheets.
• Publish web apps — either standalone or embedded in Google Sites.
• Interact with other Google services, including AdSense, Analytics, Calendar, Drive, Gmail, and Maps.
• Build add-ons to extend Google Docs, Sheets, Slides, and Forms, and publish them to the Add-on store.
• Convert an Android app into an Android add-on so that it can exchange data with a user's Google Doc or Sheet on a mobile device.
• Streamline Hangouts Chat workflows by building a chat bot.
For a quick demo of Apps Script in action, try one of our 5-minute quickstarts:
• Menus, and Custom Functions: Tailor spreadsheet functions to your needs and build reports with custom Google Sheets menu items.
• Managing Responses for Google Forms: Email Calendar invites and a personalized Google Doc to every user who responds to a form.
2 Your first script
The steps below show how to build and run a simplestandalone script that creates a Google Doc and emails you a link.
1 Set it up
1. Visit script. to open the script editor. (You'll need to be signed in to your Google account.) If this is the first time you've been to script., you'll be redirected to a page that introduces Apps Script. Click Start Scripting to proceed to the script editor.
2. A welcome screen will ask what kind of script you want to create. Click Blank Project or Close.
3. Delete any code in the script editor and paste in the code below.
templates/standalone/helloWorld.gs
VIEW ON GITHUB
/**
* Creates a Google Doc and sends an email to the current user with a link to the doc.
*/
functioncreateAndSendDocument(){
// Create a new Google Doc named 'Hello, world!'
var doc =DocumentApp.create('Hello, world!');
// Access the body of the document, then add a paragraph.
doc.getBody().appendParagraph('This document was created by Google Apps Script.');
// Get the URL of the document.
varurl=doc.getUrl();
// Get the email address of the active user - that's you.
var email =Session.getActiveUser().getEmail();
// Get the name of the document to use as an email subject line.
var subject =doc.getName();
// Append a new string to the "url" variable to use as an email body.
var body ='Link to your doc: '+url;
// Send yourself an email with a link to the document.
GmailApp.sendEmail(email, subject, body);
}
4. Select the menu item File > Save. Name your new script and click OK.
2 Try it out
1. To execute the script, either click ▶ or select the menu item Run >createAndSendDocument.
2. A dialog box will appear and tell you that the script requires authorization. Click Continue. A second dialog box will then request authorization for specific Google services. Read the notice carefully, then click Allow.
3. A yellow bar at the top of the script editor will indicate when the script is running. When the yellow bar disappears, the script has finished.
4. Check your Gmail inbox for the new email, then click the link to open the Google Doc you created.
5. Open your Google Drive. You'll see that both the script and the document are stored there.
3 Introduction to Open Source RDBMS (MySQL)
MySQL is the most popular Open Source Relational SQL Database Management System. MySQL is one of the best RDBMS being used for developing various web-based software applications. MySQL is developed, marketed and supported by MySQL AB, which is a Swedish company. This tutorial will give you a quick start to MySQL and make you comfortable with MySQL programming.
3 What is a Database?
A database is a separate application that stores a collection of data. Each database has one or more distinct APIs for creating, accessing, managing, searching and replicating the data it holds.
Other kinds of data stores can also be used, such as files on the file system or large hash tables in memory but data fetching and writing would not be so fast and easy with those type of systems.
Nowadays, we use relational database management systems (RDBMS) to store and manage huge volume of data. This is called relational database because all the data is stored into different tables and relations are established using primary keys or other keys known as Foreign Keys.
A Relational DataBase Management System (RDBMS) is a software that −
• Enables you to implement a database with tables, columns and indexes.
• Guarantees the Referential Integrity between rows of various tables.
• Updates the indexes automatically.
• Interprets an SQL query and combines information from various tables.
4 RDBMS Terminology
Before we proceed to explain the MySQL database system, let us revise a few definitions related to the database.
• Database − A database is a collection of tables, with related data.
• Table − A table is a matrix with data. A table in a database looks like a simple spreadsheet.
• Column − One column (data element) contains data of one and the same kind, for example the column postcode.
• Row − A row (= tuple, entry or record) is a group of related data, for example the data of one subscription.
• Redundancy − Storing data twice, redundantly to make the system faster.
• Primary Key − A primary key is unique. A key value can not occur twice in one table. With a key, you can only find one row.
• Foreign Key − A foreign key is the linking pin between two tables.
• Compound Key − A compound key (composite key) is a key that consists of multiple columns, because one column is not sufficiently unique.
• Index − An index in a database resembles an index at the back of a book.
• Referential Integrity − Referential Integrity makes sure that a foreign key value always points to an existing row.
5 MySQL Database
MySQL is a fast, easy-to-use RDBMS being used for many small and big businesses. MySQL is developed, marketed and supported by MySQL AB, which is a Swedish company. MySQL is becoming so popular because of many good reasons −
• MySQL is released under an open-source license. So you have nothing to pay to use it.
• MySQL is a very powerful program in its own right. It handles a large subset of the functionality of the most expensive and powerful database packages.
• MySQL uses a standard form of the well-known SQL data language.
• MySQL works on many operating systems and with many languages including PHP, PERL, C, C++, JAVA, etc.
• MySQL works very quickly and works well even with large data sets.
• MySQL is very friendly to PHP, the most appreciated language for web development.
• MySQL supports large databases, up to 50 million rows or more in a table. The default file size limit for a table is 4GB, but you can increase this (if your operating system can handle it) to a theoretical limit of 8 million terabytes (TB).
• MySQL is customizable. The open-source GPL license allows programmers to modify the MySQL software to fit their own specific environments.
All downloads for MySQL are located at MySQL Downloads. Pick the version number of MySQL Community Server which is required along with the platform you will be running it on.
6 Installing MySQL on Linux/UNIX
The recommended way to install MySQL on a Linux system is via RPM. MySQL AB makes the following RPMs available for download on its website −
• MySQL − The MySQL database server manages the databases and tables, controls user access and processes the SQL queries.
• MySQL-client − MySQL client programs, which make it possible to connect to and interact with the server.
• MySQL-devel − Libraries and header files that come in handy when compiling other programs that use MySQL.
• MySQL-shared − Shared libraries for the MySQL client.
• MySQL-bench − Benchmark and performance testing tools for the MySQL database server.
The MySQL RPMs listed here are all built on a SuSE Linux system, but they will usually work on other Linux variants with no difficulty.
Now, you will need to adhere to the steps given below, to proceed with the installation −
• Login to the system using the root user.
• Switch to the directory containing the RPMs.
• Install the MySQL database server by executing the following command. Remember to replace the filename in italics with the file name of your RPM.
[root@host]# rpm -i MySQL-5.0.9-0.i386.rpm
The above command takes care of installing the MySQL server, creating a user of MySQL, creating necessary configuration and starting the MySQL server automatically.
You can find all the MySQL related binaries in /usr/bin and /usr/sbin. All the tables and databases will be created in the /var/lib/mysql directory.
The following code box has an optional but recommended step to install the remaining RPMs in the same manner −
[root@host]# rpm -i MySQL-client-5.0.9-0.i386.rpm
[root@host]# rpm -i MySQL-devel-5.0.9-0.i386.rpm
[root@host]# rpm -i MySQL-shared-5.0.9-0.i386.rpm
[root@host]# rpm -i MySQL-bench-5.0.9-0.i386.rpm
7 Installing MySQL on Windows
The default installation on any version of Windows is now much easier than it used to be, as MySQL now comes neatly packaged with an installer. Simply download the installer package, unzip it anywhere and run the setup.exe file.
The default installer setup.exe will walk you through the trivial process and by default will install everything under C:\mysql.
Test the server by firing it up from the command prompt the first time. Go to the location of the mysqld server which is probably C:\mysql\bin, and type −
mysqld.exe --console
NOTE − If you are on NT, then you will have to use mysqld-nt.exe instead of mysqld.exe
If all went well, you will see some messages about startup and InnoDB. If not, you may have a permissions issue. Make sure that the directory that holds your data is accessible to whatever user (probably MySQL) the database processes run under.
MySQL will not add itself to the start menu, and there is no particularly nice GUI way to stop the server either. Therefore, if you tend to start the server by double clicking the mysqld executable, you should remember to halt the process by hand by using mysqladmin, Task List, Task Manager, or other Windows-specific means.
8 Verifying MySQL Installation
After MySQL, has been successfully installed, the base tables have been initialized and the server has been started: you can verify that everything is working as it should be via some simple tests.
1 Use the mysqladmin Utility to Obtain Server Status
Use mysqladmin binary to check the server version. This binary would be available in /usr/bin on linux and in C:\mysql\bin on windows.
[root@host]#mysqladmin --version
It will produce the following result on Linux. It may vary depending on your installation −
mysqladminVer 8.23 Distrib 5.0.9-0, for redhat-linux-gnu on i386
If you do not get such a message, then there may be some problem in your installation and you would need some help to fix it.
2 Execute simple SQL commands using the MySQL Client
You can connect to your MySQL server through the MySQL client and by using the mysql command. At this moment, you do not need to give any password as by default it will be set as blank.
You can just use following command −
[root@host]#mysql
It should be rewarded with a mysql> prompt. Now, you are connected to the MySQL server and you can execute all the SQL commands at the mysql> prompt as follows −
mysql> SHOW DATABASES;
+----------+
| Database |
+----------+
| mysql |
| test |
+----------+
2 rows in set (0.13 sec)
9 Post-installation Steps
MySQL ships with a blank password for the root MySQL user. As soon as you have successfully installed the database and the client, you need to set a root password as given in the following code block −
[root@host]#mysqladmin -u root password "new_password";
Now to make a connection to your MySQL server, you would have to use the following command −
[root@host]#mysql -u root -p
Enter password:*******
UNIX users will also want to put your MySQL directory in your PATH, so you won't have to keep typing out the full path everytime you want to use the command-line client.
For bash, it would be something like −
export PATH = $PATH:/usr/bin:/usr/sbin
10 Running MySQL at Boot Time
If you want to run the MySQL server at boot time, then make sure you have the following entry in the /etc/rc.local file.
/etc/init.d/mysqld start
Also,you should have the mysqld binary in the /etc/init.d/ directory.
PostgreSQL Tutorial
PostgreSQL is a powerful, open source object-relational database system. It has more than 15 years of active development and a proven architecture that has earned it a strong reputation for reliability, data integrity, and correctness. PostgreSQL runs on all major operating systems, including Linux, UNIX (AIX, BSD, HP-UX, SGI IRIX, Mac OS X, Solaris, Tru64), and Windows. This tutorial will give you quick start with PostgreSQL and make you comfortable with PostgreSQL programming.
PostgreSQL is a powerful, open source object-relational database system. It has more than 15 years of active development phase and a proven architecture that has earned it a strong reputation for reliability, data integrity, and correctness.
1 What is PostgreSQL?
PostgreSQL (pronounced as post-gress-Q-L) is an open source relational database management system (DBMS) developed by a worldwide team of volunteers. PostgreSQL is not controlled by any corporation or other private entity and the source code is available free of charge.
1 A Brief History of PostgreSQL
PostgreSQL, originally called Postgres, was created at UCB by a computer science professor named Michael Stonebraker. Stonebraker started Postgres in 1986 as a follow-up project to its predecessor, Ingres, now owned by Computer Associates.
• 1977-1985 − A project called INGRES was developed.
o Proof-of-concept for relational databases
o Established the company Ingres in 1980
o Bought by Computer Associates in 1994
• 1986-1994 − POSTGRES
o Development of the concepts in INGRES with a focus on object orientation and the query language - Quel
o The code base of INGRES was not used as a basis for POSTGRES
o Commercialized as Illustra (bought by Informix, bought by IBM)
• 1994-1995 − Postgres95
o Support for SQL was added in 1994
o Released as Postgres95 in 1995
o Re-released as PostgreSQL 6.0 in 1996
o Establishment of the PostgreSQL Global Development Team
2 Key Features of PostgreSQL
PostgreSQL runs on all major operating systems, including Linux, UNIX (AIX, BSD, HP-UX, SGI IRIX, Mac OS X, Solaris, Tru64), and Windows. It supports text, images, sounds, and video, and includes programming interfaces for C / C++, Java, Perl, Python, Ruby, Tcl and Open Database Connectivity (ODBC).
PostgreSQL supports a large part of the SQL standard and offers many modern features including the following −
• Complex SQL queries
• SQL Sub-selects
• Foreign keys
• Trigger
• Views
• Transactions
• Multiversion concurrency control (MVCC)
• Streaming Replication (as of 9.0)
• Hot Standby (as of 9.0)
You can check official documentation of PostgreSQL to understand the above-mentioned features. PostgreSQL can be extended by the user in many ways. For example by adding new −
• Data types
• Functions
• Operators
• Aggregate functions
• Index methods
3 Procedural Languages Support
PostgreSQL supports four standard procedural languages, which allows the users to write their own code in any of the languages and it can be executed by PostgreSQL database server. These procedural languages are - PL/pgSQL, PL/Tcl, PL/Perl and PL/Python. Besides, other non-standard procedural languages like PL/PHP, PL/V8, PL/Ruby, PL/Java, etc., are also supported.
NoSQL Databases
We know that MongoDB is a NoSQL Database, so it is very necessary to know about NoSQL Database to understand MongoDB throughly .
1 What is NoSQL Database
Databases can be divided in 3 types:
1. RDBMS (Relational Database Management System)
2. OLAP (Online Analytical Processing)
3. NoSQL (recently developed database)
2 NoSQL Database
NoSQL Database is used to refer a non-SQL or non relational database.
It provides a mechanism for storage and retrieval of data other than tabular relations model used in relational databases. NoSQL database doesn't use tables for storing data. It is generally used to store big data and real-time web applications.
3 History behind the creation of NoSQL Databases
In the early 1970, Flat File Systems are used. Data were stored in flat files and the biggest problems with flat files are each company implement their own flat files and there are no standards. It is very difficult to store data in the files, retrieve data from files because there is no standard way to store data.
Then the relational database was created by E.F. Codd and these databases answered the question of having no standard way to store data. But later relational database also get a problem that it could not handle big data, due to this problem there was a need of database which can handle every types of problems then NoSQL database was developed.
4 Advantages of NoSQL
o It supports query language.
o It provides fast performance.
o It provides horizontal scalability.
MongoDB Tutorial
MongoDB is an open-source document database and leading NoSQL database. MongoDB is written in C++. This tutorial will give you great understanding on MongoDB concepts needed to create and deploy a highly scalable and performance-oriented database.
MongoDB is a cross-platform, document oriented database that provides, high performance, high availability, and easy scalability. MongoDB works on concept of collection and document.
1 Database
Database is a physical container for collections. Each database gets its own set of files on the file system. A single MongoDB server typically has multiple databases.
2 Collection
Collection is a group of MongoDB documents. It is the equivalent of an RDBMS table. A collection exists within a single database. Collections do not enforce a schema. Documents within a collection can have different fields. Typically, all documents in a collection are of similar or related purpose.
3 Document
A document is a set of key-value pairs. Documents have dynamic schema. Dynamic schema means that documents in the same collection do not need to have the same set of fields or structure, and common fields in a collection's documents may hold different types of data.
The following table shows the relationship of RDBMS terminology with MongoDB.
|RDBMS |MongoDB |
|Database |Database |
|Table |Collection |
|Tuple/Row |Document |
|Column |Field |
|Table Join |Embedded Documents |
|Primary Key |Primary Key (Default key _id provided by mongodb itself) |
|Database Server and Client |
|Mysqld/Oracle |mongod |
|mysql/sqlplus |mongo |
4 Sample Document
Following example shows the document structure of a blog site, which is simply a comma separated key value pair.
{
_id:ObjectId(7df78ad8902c)
title:'MongoDB Overview',
description:'MongoDB is no sql database',
by:'tutorials point',
url:'',
tags:['mongodb','database','NoSQL'],
likes:100,
comments:[
{
user:'user1',
message:'My first comment',
dateCreated:newDate(2011,1,20,2,15),
like:0
},
{
user:'user2',
message:'My second comments',
dateCreated:newDate(2011,1,25,7,45),
like:5
}
]
}
_id is a 12 bytes hexadecimal number which assures the uniqueness of every document. You can provide _id while inserting the document. If you don’t provide then MongoDB provides a unique id for every document. These 12 bytes first 4 bytes for the current timestamp, next 3 bytes for machine id, next 2 bytes for process id of MongoDB server and remaining 3 bytes are simple incremental VALUE.
MongoDB - Advantages
Any relational database has a typical schema design that shows number of tables and the relationship between these tables. While in MongoDB, there is no concept of relationship.
1 Advantages of MongoDB over RDBMS
• Schema less − MongoDB is a document database in which one collection holds different documents. Number of fields, content and size of the document can differ from one document to another.
• Structure of a single object is clear.
• No complex joins.
• Deep query-ability. MongoDB supports dynamic queries on documents using a document-based query language that's nearly as powerful as SQL.
• Tuning.
• Ease of scale-out − MongoDB is easy to scale.
• Conversion/mapping of application objects to database objects not needed.
• Uses internal memory for storing the (windowed) working set, enabling faster access of data.
2 Why Use MongoDB?
• Document Oriented Storage − Data is stored in the form of JSON style documents.
• Index on any attribute
• Replication and high availability
• Auto-sharding
• Rich queries
• Fast in-place updates
• Professional support by MongoDB
3 Where to Use MongoDB?
• Big Data
• Content Management and Delivery
• Mobile and Social Infrastructure
• User Data Management
• Data Hub
Hadoop Tutorial
Hadoop is an open-source framework that allows to store and process big data in a distributed environment across clusters of computers using simple programming models. It is designed to scale up from single servers to thousands of machines, each offering local computation and storage.
Hadoop is an Apache open source framework written in java that allows distributed processing of large datasets across clusters of computers using simple programming models. The Hadoop framework application works in an environment that provides distributed storage and computation across clusters of computers. Hadoop is designed to scale up from single server to thousands of machines, each offering local computation and storage.
1 Hadoop Architecture
At its core, Hadoop has two major layers namely −
• Processing/Computation layer (MapReduce), and
• Storage layer (Hadoop Distributed File System).
[pic]
2 MapReduce
MapReduce is a parallel programming model for writing distributed applications devised at Google for efficient processing of large amounts of data (multi-terabyte data-sets), on large clusters (thousands of nodes) of commodity hardware in a reliable, fault-tolerant manner. The MapReduce program runs on Hadoop which is an Apache open-source framework.
3 Hadoop Distributed File System
The Hadoop Distributed File System (HDFS) is based on the Google File System (GFS) and provides a distributed file system that is designed to run on commodity hardware. It has many similarities with existing distributed file systems. However, the differences from other distributed file systems are significant. It is highly fault-tolerant and is designed to be deployed on low-cost hardware. It provides high throughput access to application data and is suitable for applications having large datasets.
Apart from the above-mentioned two core components, Hadoop framework also includes the following two modules −
• Hadoop Common − These are Java libraries and utilities required by other Hadoop modules.
• Hadoop YARN − This is a framework for job scheduling and cluster resource management.
4 How Does Hadoop Work?
It is quite expensive to build bigger servers with heavy configurations that handle large scale processing, but as an alternative, you can tie together many commodity computers with single-CPU, as a single functional distributed system and practically, the clustered machines can read the dataset in parallel and provide a much higher throughput. Moreover, it is cheaper than one high-end server. So this is the first motivational factor behind using Hadoop that it runs across clustered and low-cost machines.
Hadoop runs code across a cluster of computers. This process includes the following core tasks that Hadoop performs −
• Data is initially divided into directories and files. Files are divided into uniform sized blocks of 128M and 64M (preferably 128M).
• These files are then distributed across various cluster nodes for further processing.
• HDFS, being on top of the local file system, supervises the processing.
• Blocks are replicated for handling hardware failure.
• Checking that the code was executed successfully.
• Performing the sort that takes place between the map and reduce stages.
• Sending the sorted data to a certain computer.
• Writing the debugging logs for each job.
5 Advantages of Hadoop
• Hadoop framework allows the user to quickly write and test distributed systems. It is efficient, and it automatic distributes the data and work across the machines and in turn, utilizes the underlying parallelism of the CPU cores.
• Hadoop does not rely on hardware to provide fault-tolerance and high availability (FTHA), rather Hadoop library itself has been designed to detect and handle failures at the application layer.
• Servers can be added or removed from the cluster dynamically and Hadoop continues to operate without interruption.
• Another big advantage of Hadoop is that apart from being open source, it is compatible on all the platforms since it is Java based.
Unit 4
MySQL is the most popular database system used with PHP.
[pic]
What is MySQL?
• MySQL is a database system used on the web
• MySQL is a database system that runs on a server
• MySQL is ideal for both small and large applications
• MySQL is very fast, reliable, and easy to use
• MySQL uses standard SQL
• MySQL compiles on a number of platforms
• MySQL is free to download and use
• MySQL is developed, distributed, and supported by Oracle Corporation
• MySQL is named after co-founder Monty Widenius's daughter: My
The data in a MySQL database are stored in tables. A table is a collection of related data, and it consists of columns and rows.
Databases are useful for storing information categorically. A company may have a database with the following tables:
• Employees
• Products
• Customers
• Orders
PHP + MySQL Database System
• PHP combined with MySQL are cross-platform (you can develop in Windows and serve on a Unix platform)
[pic]
Database Queries
A query is a question or a request.
We can query a database for specific information and have a recordset returned.
Look at the following query (using standard SQL):
SELECT LastName FROM Employees
The query above selects all the data in the "LastName" column from the "Employees" table.
To learn more about SQL, please visit our SQL tutorial.
[pic]
Download MySQL Database
If you don't have a PHP server with a MySQL Database, you can download it for free here:
[pic]
Facts About MySQL Database
MySQL is the de-facto standard database system for web sites with HUGE volumes of both data and end-users (like Facebook, Twitter, and Wikipedia).
Another great thing about MySQL is that it can be scaled down to support embedded database applications.
Look at for an overview of companies using MySQL.
PHP Connect to MySQL
PHP 5 and later can work with a MySQL database using:
• MySQLi extension (the "i" stands for improved)
• PDO (PHP Data Objects)
Earlier versions of PHP used the MySQL extension. However, this extension was deprecated in 2012.
[pic]
1 Should I Use MySQLi or PDO?
If you need a short answer, it would be "Whatever you like".
Both MySQLi and PDO have their advantages:
PDO will work on 12 different database systems, whereas MySQLi will only work with MySQL databases.
So, if you have to switch your project to use another database, PDO makes the process easy. You only have to change the connection string and a few queries. With MySQLi, you will need to rewrite the entire code - queries included.
Both are object-oriented, but MySQLi also offers a procedural API.
Both support Prepared Statements. Prepared Statements protect from SQL injection, and are very important for web application security.
[pic]
2 MySQL Examples in Both MySQLi and PDO Syntax
In this, and in the following chapters we demonstrate three ways of working with PHP and MySQL:
• MySQLi (object-oriented)
• MySQLi (procedural)
• PDO
[pic]
3 MySQLi Installation
For Linux and Windows: The MySQLi extension is automatically installed in most cases, when php5 mysql package is installed.
For installation details, go to:
[pic]
4 PDO Installation
For installation details, go to:
5 Open a Connection to MySQL
Before we can access data in the MySQL database, we need to be able to connect to the server:
1 Example (MySQLi Object-Oriented)
2 Example (MySQLi Procedural)
3 Example (PDO)
6 Close the Connection
The connection will be closed automatically when the script ends. To close the connection before, use the following:
1 Example (MySQLi Object-Oriented)
$conn->close();
2 Example (MySQLi Procedural)
mysqli_close($conn);
3 Example (PDO)
$conn = null;
MySQL SHOW TABLES: List Tables In a MySQL Database
To list tables in a MySQL database, you follow these steps:
1. Login to the MySQL database server using a MySQL client such as mysql
2. Switch to a specific database using the USE statement.
3. Use the SHOW TABLES command.
The following illustrates the syntax of the MySQL SHOW TABLES command:
|1|SHOWTABLES; |
1 MySQL SHOW TABLES examples
The following example shows you how to list the table in the classicmodels database.
Step 1. Connect to the MySQL database server:
|1|>mysql-uroot-p |
|2|Enterpassword:********** |
|3|mysql> |
Step 2. Switch to classicmodels database:
|1|mysql>useclassicmodels; |
|2|Databasechanged |
|3|mysql> |
Step 3. Show tables in the classicmodels database:
|1 |>showtables; |
|2 |+-------------------------+ |
|3 ||Tables_in_classicmodels| |
|4 |+-------------------------+ |
|5 ||customers | |
|6 ||employees | |
|7 ||offices | |
|8 ||orderdetails | |
|9 ||orders | |
|10 ||payments | |
|11 ||productlines | |
|12 ||products | |
|13 |+-------------------------+ |
|14 |8rowsinset(0.00sec) |
The SHOW TABLES command allows you to show if a table is a base table or a view. To include the table type in the result, you use the following form of the SHOW TABLES statement.
|1|SHOWFULLTABLES; |
Let’s create a view in the classicmodels database called contacts that includes first name, last name and phone from the employees and customers tables for the demonstration.
|1|CREATEVIEWcontacts |
|2|AS |
|3|SELECTlastName,firstName,extensionasphone |
|4|FROMemployees |
|5|UNION |
|6|SELECTcontactFirstName,contactLastName,phone |
|7|FROMcustomers; |
Now, you issue the SHOW FULL TABLES command:
|1 |>SHOWFULLTABLES |
|2 |+-------------------------+------------+ |
|3 ||Tables_in_classicmodels|Table_type| |
|4 |+-------------------------+------------+ |
|5 ||contacts |VIEW | |
|6 ||customers |BASETABLE| |
|7 ||employees |BASETABLE| |
|8 ||offices |BASETABLE| |
|9 ||orderdetails |BASETABLE| |
|10 ||orders |BASETABLE| |
|11 ||payments |BASETABLE| |
|12 ||productlines |BASETABLE| |
|13 ||products |BASETABLE| |
|14 |+-------------------------+------------+ |
|15 |9rowsinset(0.00sec) |
As you can see, all the tables are the base tables except for the contacts table which is a view.
For the database that has many tables, showing all tables at a time may not be intuitive.
Fortunately, the SHOW TABLES command provides you with an option that allows you to filter the returned tables using the LIKE operator or an expression in the WHERE clause as follows:
|1|SHOWTABLESLIKEpattern; |
|2| |
|3|SHOWTABLESWHEREexpression; |
For example, to shows all tables in the classicmodels database that start with the letter p, you use the following statement:
|1|>SHOWTABLESLIKE'p%'; |
|2|+------------------------------+ |
|3||Tables_in_classicmodels(p%)| |
|4|+------------------------------+ |
|5||payments | |
|6||productlines | |
|7||products | |
|8|+------------------------------+ |
|9|3rowsinset(0.00sec) |
Or to show the tables that end with the string 'es', you use the following statement:
|1|>SHOWTABLESLIKE'%es'; |
|2|+-------------------------------+ |
|3||Tables_in_classicmodels(%es)| |
|4|+-------------------------------+ |
|5||employees | |
|6||offices | |
|7||productlines | |
|8|+-------------------------------+ |
|9|3rowsinset(0.00sec) |
The following statement illustrates how to use the WHERE clause in the SHOW TABLES statement to list all the views in the classicmodels database.
|1|>SHOWFULLTABLESWHEREtable_type='VIEW'; |
|2|+-------------------------+------------+ |
|3||Tables_in_classicmodels|Table_type| |
|4|+-------------------------+------------+ |
|5||contacts |VIEW | |
|6|+-------------------------+------------+ |
|7|1rowinset(0.00sec) |
Sometimes, you want to see the tables in the database that you are not connected to. In this case, you can use the FROM clause of the SHOW TABLES statement to specify the database from which you want to show the tables.
The following example demonstrates how to show tables that start with 'time';
|1 |>SHOWTABLESFROMmysqlLIKE'time%'; |
|2 |+---------------------------+ |
|3 ||Tables_in_mysql(time%) | |
|4 |+---------------------------+ |
|5 ||time_zone | |
|6 ||time_zone_leap_second | |
|7 ||time_zone_name | |
|8 ||time_zone_transition | |
|9 ||time_zone_transition_type| |
|10 |+---------------------------+ |
|11 |5rowsinset(0.00sec) |
The following statement is equivalent to the statement above but it uses IN instead of FROM.
|1|SHOWTABLESINmysqlLIKE'time%'; |
It’s important to note that if you don’t have privileges for a base table or view, it won’t show up in the result set of the SHOW TABLES command.
1 The USE Command in MySQL
The syntax for the USE command is:
mysql>
For example, this code switches to the database named "Dresses."
mysql>
After you select a database, it remains the default until you end the session or choose another database with the USE command.
2 Identifying the Current Database
If you are unsure which database is currently in use, use the following code:
This code returns the name of the database currently in use. If no database is currently in use, it returns NULL.
PHP Create a MySQL Database
A database consists of one or more tables.
You will need special CREATE privileges to create or to delete a MySQL database.
[pic]
1 Create a MySQL Database Using MySQLi and PDO
The CREATE DATABASE statement is used to create a database in MySQL.
The following examples create a database named "myDB":
1 Example (MySQLi Object-oriented)
PHP Create MySQL Tables
A database table has its own unique name and consists of columns and rows.
[pic]
Create a MySQL Table Using MySQLi and PDO
The CREATE TABLE statement is used to create a table in MySQL.
We will create a table named "MyGuests", with five columns: "id", "firstname", "lastname", "email" and "reg_date":
CREATE TABLE MyGuests (
id INT(6) UNSIGNED AUTO_INCREMENT PRIMARY KEY,
firstname VARCHAR(30) NOT NULL,
lastname VARCHAR(30) NOT NULL,
email VARCHAR(50),
reg_date TIMESTAMP
)
The data type specifies what type of data the column can hold. For a complete reference of all the available data types, go to our Data Types reference.
After the data type, you can specify other optional attributes for each column:
• NOT NULL - Each row must contain a value for that column, null values are not allowed
• DEFAULT value - Set a default value that is added when no other value is passed
• UNSIGNED - Used for number types, limits the stored data to positive numbers and zero
• AUTO INCREMENT - MySQL automatically increases the value of the field by 1 each time a new record is added
• PRIMARY KEY - Used to uniquely identify the rows in a table. The column with PRIMARY KEY setting is often an ID number, and is often used with AUTO_INCREMENT
Each table should have a primary key column (in this case: the "id" column). Its value must be unique for each record in the table.
The following examples shows how to create the table in PHP:
1 Example (MySQLi Object-oriented)
PHP Select Data From MySQL
The SELECT statement is used to select data from one or more tables:
SELECT column_name(s) FROM table_name
or we can use the * character to select ALL columns from a table:
SELECT * FROM table_name
1 Select Data WithMySQLi
The following example selects the id, firstname and lastname columns from the MyGuests table and displays it on the page:
1 Example (MySQLi Object-oriented)
Code lines to explain from the example above:
First, we set up an SQL query that selects the id, firstname and lastname columns from the MyGuests table. The next line of code runs the query and puts the resulting data into a variable called $result.
Then, the function num_rows() checks if there are more than zero rows returned.
If there are more than zero rows returned, the function fetch_assoc() puts all the results into an associative array that we can loop through. The while() loop loops through the result set and outputs the data from the id, firstname and lastname columns.
PHP Insert Data Into MySQL
1 Insert Data Into MySQL Using MySQLi and PDO
After a database and a table have been created, we can start adding data in them.
Here are some syntax rules to follow:
• The SQL query must be quoted in PHP
• String values inside the SQL query must be quoted
• Numeric values must not be quoted
• The word NULL must not be quoted
The INSERT INTO statement is used to add new records to a MySQL table:
INSERT INTO table_name (column1, column2, column3,...)
VALUES (value1, value2, value3,...)
To learn more about SQL, please visit our SQL tutorial.
In the previous chapter we created an empty table named "MyGuests" with five columns: "id", "firstname", "lastname", "email" and "reg_date". Now, let us fill the table with data.
Note: If a column is AUTO_INCREMENT (like the "id" column) or TIMESTAMP (like the "reg_date" column), it is no need to be specified in the SQL query; MySQL will automatically add the value.
The following examples add a new record to the "MyGuests" table:
1 Example (MySQLi Object-oriented)
................
................
In order to avoid copyright disputes, this page is only a partial summary.
To fulfill the demand for quickly locating and searching documents.
It is intelligent file search solution for home and business.
Related searches
- wordpress passing data between pages
- wordpress business templates
- wordpress rss feed not working
- wordpress jquery is not defined
- create wordpress blog
- wordpress roles editor
- wordpress full rss feed
- wordpress rss feed settings
- wordpress rss feed plugin
- wordpress display rss feed
- wordpress rss feed link
- wordpress rss feed to post