What is Pre-Process De-Duplication?

October 14, 2015 Stewart Parkin

The context of Pre-Process De-duplication is also known as Source De-duplication in most case. The source de-duplication is mostly used for de-duplication of data prior to its transmission to the device for storage. The entire data are channelled via the source de-dupe software or hardware before being ready to be transmitted to the storage device where it will be stored. The major objective of source de-duplication is to prevent sending of duplicated data across the network to the device where it will be stored. There is establishment of connection using the designated storage device as well as evaluation of data prior to the time when it will initiate the de-duplication process. The synchronisation with the target disk upholds all through the process in order to ensure synchronisation of data, removing the files that match at the source. The main advantage of this is that it helps save bandwidth for the user.

In order to identify changed bytes, there is always the need for byte level scans by either the source de-dupe software or hardware. To make recovery easy for the user, the changed bytes are transferred to the destination or target device, pointing it to the original indexes and files updated with the pointer. Indeed, it does not take time to control the entire operation as they happen quickly without compromising the accuracy and efficiency of the process. The process of source de-dupe is light on processing power when compared to post process de-dupe. It has been observed that source de-duplication has the capability to categorise data in real time. The device configurations that are based on policy can classify data at granular levels, as well as filter out data while they pass across the source de-dupe device. There can be addition or removal of files on usual basis of the group, domain, user, owner, age, path, file type, or storage type, or even on the basis of RPO or retention periods.

Having said the advantages of source de-dupe, there are some disadvantages associated with source de-dupe. It is true that source de-duplication helps decrease the bandwidth you need to transmit data or files to the destination or target, however, there is imposition of higher processing load on the clients, as the entire process is involved in the source de-duplication. In addition, the central processing unit (CPU) power consumption of your device will go higher by about 25% to 50% during source de-duplication process, which may not really be favourable for you at all. There may be needs to incorporate source based de-dupe nodes into each of the locations connected. This involves more cost and will obviously be more expensive than the target de-duplication techniques, where all the de-duplications are carried out on one de-duplication device, within the network nodal point.

Lastly, if the existing software does not support de-duplication hardware or algorithms, there may be the need for redesigning of the software. This, however, is not a problem in target de-duplication, where there is isolation of de-dupe hardware and software from the organisation’s hardware or software. Also, there are no changes needed at the source de-dupe.

Cloud Computing Compliance and Industry Standards

October 12, 2015 Kris Price

Though cloud computing is known as a trendy technology, it is not in any way new in the computer networking field. The truth is that telecommunication companies were the first cloud service providers and they established some expectations for customers as well as industry standards for the past decades. Fast forward to 2015, the expectations remain unchanged; the standards of the cloud computing industry has been similarly modelled as in the Telcos, with some additions that are triggered by nature of operations and the transaction types that occur in the new cloud applications.

Definition of standard may be linked to the type of cloud computing in use. The cloud may be public, hybrid or private clouds. Also, the mandate around the private clouds requires loyalty to defined process and well established control frameworks. Documentation, backup controls, and Service Level Agreements (SLAs), as well as infrastructure libraries for service management and business integration must be made available. In the public cloud, there must be adherence to established frameworks as well provision for audits under various compliance acts.

Standards adopted may be determined based on the type of industry. The industrial standard may be dictated by the country or segment of operation and industrial vertical. For instance, the health care industry will be governed by HIPAA, while Sarbanes Oxley governs the financial industry in the USA and Canada. There may be sets of requirements and regulation by the industry itself for access control, security, protection of data, segregation of duties and lots more. The requirements for SLA may differ from one segment to another based on the amount of data processed, company size, and the kind of data. The regulation of the country, where the the company (or industry and/or service provided) is located may have an influence on the kind of standards adopted. Current tools and the available processes will also affect the standards adopted for the cloud. Implemented standards. You will need to consider customers and/or industry practises inside and outside the company / industry at the same time to refine the standards that should be implemented.

The current standards or requirements already established by vendors globally will have a huge impact on the standardisations and expectations. This is always obvious when software usage and licensing is involved. Industries need to always be mindful of the restrictions as well as the components that are not defined clearly; and also create practises that will not lead to unforeseen expenses, any kind of violations or breaches. For the fact that cloud is a combination of variety of technologies, standards need to be followed strictly, along with integrated compliance standards that may come up in the context.

In fact, cloud computing service providers must always investigate important areas for compliance and determine the minimum expectations from the regulatory authorities, clients, and competitors within the industry vertical.

Effects of Bandwidth in Cloud Computing

October 9, 2015 Rob Mackle

The term “bandwidth” has been used in Electrical Engineering for years to mean “the difference between the upper and lower frequencies in a continuous set of frequencies, measured in hertz”. In the early 1990’s Telcos started to use the term “bandwidth” to describe the volume of data handled and defined it as “the transmission rate of data across a network”. Bandwidth in data transmission is measured in bits per second and it represents the capacity of network connection. Increase in the capacity means improved performance, considering other factors like latency. We will further discuss the effects of utilisation of bandwidth to the challenges associated with cloud computing.

Cloud computing providers usually calculate required bandwidth of customers just by considering the available quantity of bandwidth as well as the mean bandwidth utilisation needed by variety of applications. In addition, cloud computing providers consider latencies in transmission to calculate the required time to upload both the initial backup and all subsequent backups. For that reason, Internet based cloud backup service providers work hard to enhance the overall Internet bandwidth. They also do everything within their power to reduce the amount of data that flows through their pipes. There are many things the cloud service providers do to achieve such goals. They can use incremental backup technologies, link load balancing technologies or even some exceptional binary patching to transmit and extract file’s changes so as to reduce/ balance the transmitted amount of data. In addition, both de-duplication and file compression techniques may be used to decrease the quantity of files that are transmitted over the network.

Information Technology administrators are often advised to determine, as accurately as possible, the quantity of bandwidth that will be needed by the organisation for both data storage and transfer operations in the cloud, and also the latency, expressed in milliseconds. Therefore, they have to consider the number of users and systems that will be responsible for pushing data into the available network space for data storage and other functions at non-peak and peak hours.

The online backup that is built with self-service enabling features, augmented by administrative interface that is user-friendly can easily be built with tools that allow customers to select the things that can be backed up at any point in time. There is also inbuilt filter, which enables users to add or remove files as well as folders from their sets of backups. There may be scheduling of backup sets for upload at different points in time in the Internet based servers. Massive data transfers can be scheduled during off-peak times of the organisation, increasing the speed of the transfer of data to when there is enough bandwidth.

Nevertheless, it is important for you to know that there is interconnection between bandwidth and latency. The connection speed of the network is the function of latency and bandwidth. Latency cannot be decreased drastically, however, bandwidth can be increased at anytime.

Making Use of SOX as a Regulatory Intervention

October 7, 2015 Callum Huddlestone

Sarbanes Oxley (SOX) is among the regulatory intervention controls, which raucously promotes the finance internal control functions. SOX also makes such finance controls very transparent. The connection between risk management and internal audit is stressed and the controls are known to be liable for provision of required guarantee that the management is practically justifying and identifying all the possible risks that may come up from the internal systems, business operations, as well as organisational structure. SOX is the force behind management readiness and evaluation of controls in an organisation. That is why Sarbanes Oxley is among most reorganised interventions used in an organisation.

Due to the benefits of SOX in an organisation, industries that act in accordance with the terms put in more efforts to find out risks involved in their organisation and completely check the associated risks. Very little or no human intervention is required, as the entire processes and internal controls are known to be automated. There are lots of things involved in the process starting from documentation, evaluation as well as standardisation of controls in the entire enterprise for overall enhancement and effectiveness. There is establishment of automated control measures which help to determine the overall continuous improvement and effectiveness.

Nevertheless, the unique lack of regulatory guidance associated with SOX has been often raised by many organisations that have utilised the system. For that reason, there is a limitation in the use of purpose-driven applications, making information to always be soiled, fragmented as well as scattered in the entire organisations. These have led to deficiencies in controls as only about 20-50% of controls are automated. On the other hand, the manual controls have turned to show themselves as being expensive and more labour intensive. There is also a problem in the workflows, which are often seen to be uncoordinated, as well as deficiencies in the coordination existing in the midst of external auditors and the internal compliance teams leading to more expensive consulting costs.

So, for enablement and reliability purposes, there is recommendation for automated system that is content based. The ability of an enterprise to standardise controls, document contents, and effectively manage compliance process is said to be the most important for a complete enhancement of controls. The critical sources for SOX compliance are, therefore, said to be the collaborative content and document management systems. The rule-based security models and workflow management, which monitor or sign off on the assigned tasks in the entire organisation, are said to be enabled by these systems. There is also improvement in the preventive controls, and the enforcement of segmentation of duties by user management systems. These help to guarantee sustainability, integrity, and persistency of the generated, used and stored data by the enterprise for compliance of SOX.

Therefore, it is important for you to know that pulling off SOX is complicated and can only be achieved through technology integration to function in one accord, so as to manage risk effectively. What is required for this to be possible is just hard work, and nothing less than that.

The Power of Activity Logs in Cloud Backup Services

October 5, 2015 Damien Garvey

Since your online data backup is hosted at a third party company, there are chances that you will have oodles of questions for your online backup provider. Do you know that your data can easily be compromised before it can actually reach the vaults of your online back provider? How are you sure that your data is adequately and securely stored in their vault once transmitted? What exactly make you think that your data will continue to be safe in the Internet-based datacentre? These and more are what you need to think before hiring an online backup service provider.

Majority online backup providers normally send manual or automated alerts to their customers at different stages of their data backup recovery or storage processes so as to pro-actively manage the anxiety of their customers. The main purpose of customer alert, which most online backup providers normally send to customers, is to inform them that something was not done properly the way it should have been done, so that they can either contact the service provider immediately or attend to the problem at their own end. The providers typically configure the alert when the customer first subscribes for the service. So, the client’s machine is installed on the network by the software agent and that helps to check the status of the service repeatedly and generates alert for the benefit of the provider and the customer.

One popular type of customer alert is simply known as logs. The logs are known to document the steps on the executed process. Also, all activities on the system are known to be supported by logs. There are backup logs, which inform the customer whether the backup process was smoothly done or there is/are error(s) that prevented it from being completed. If there is failed backups, the administrator will be able to notice it the moment (s)he logs into the system, as it will pop up in the form of a message or a report.

Furthermore, the user logs track and control the activities of the users and check against unauthorised attempts to access the system with the use of authenticated identity of the user. Also, passwords are recorded on the user logs, and required alerts can be generated. There may be other logs that are used to track behaviour on the network.

Custom alerts are being used to keep track of the system’s functionalities for most cloud backup service providers. Indeed, the alerts are designed to point out problems that need solutions so as to avoid problems in the system. So, the backup providers may just resolve the issues at their end without the need to contact the customer. Alternately, the service provider could notify the customer through email to handle the issue at their end. If the problem requires advanced knowledge to solve it, then the service provider may send specialised technical experts to guide the customer to resolve the issue.

Points to Note about Bare Metal Restore

October 2, 2015 Stewart Parkin

In general, majority of data restore systems often specify their ability to restore data to only the computing systems with similar or the same software running on them (i.e., environment like operating system, applications, and others). But the case of Base Metal restore is totally different and more unique as it is a repair and restoration technique designed to restore data to a computing system with entirely different hardware and software environment. So, the restoration process is coined “Bare Metal”, which means that the process does not need any requirement from the computing system where the data backup was done for it to restore the data.

Some Difference between Bare Metals and Disk Image Restore
Though, Bare Metal and Disk image restores work with similar process with each other, there are differences that exist between them. While Bare Metal involved installation of operating system, applications and component of data in the image file, Disk Image Restore only requires the installation of the restoration software and a copy of the disk image installed in the backed up computing system. With the help of Bare Metal imaging applications, the process of bare metal restore will be made easier. There is always replication of images of software components as well as the data to the new disk that were present at the time of backup through snapshots. This normally makes it easy for the data and the images to be restored using Bare Metal. The entire file systems in the disk images as well as disk partitions are copied using Linux Booth CD. There is also creation of new partition with either the same or larger size disk image, which resulted from the backup. Some servers support Bare Metal restores, and users can make use of such Bare Metal enabled servers to restore or recover their system in to a HyperV virtual machine.

Indeed, just with one single restore process, the entire computing system can be restored to the last known good state with the help of Bare Metal systems. This wonderful system is great and useful for recovering of data from the system during a disaster. This is made possible owing to the fact that the disk image can be useful for recreating the whole system at completely new geo-location with new hardware, using online application access. The restoration process involved in Bare Metal has been referred to as an art instead of a science, as it differs in the implementation from other system restore software. Not all online backup providers have Bare Metal services. Asigra is a powerful cloud backup software that has the capability to provide Bare Metal restore services.

Before starting your restoration using Bare Metal, you must ensure that the configuration of your system is supported by bare metal as the storage configurations of the servers may not be the same. So, confirm whether the basic partition table type of your system is supported by Bare Metal before going ahead to make use of it for your system data restore.