VMWare vStorage APIs, Hot Add, iSCSI and Data Recovery

vStorage APIs

Perhaps the biggest benefit to backups and storage in vSphere is the new vStorage APIs that VMware developed. These APIs allow third-party applications to directly interface with the VMkernel without the need for scripts or agents. The vStorage APIs existed in VI3, but were referred to as the VMware Consolidated Backup (VCB) Backup Framework. However, unlike VMware Consolidated Backup, they are not a separate standalone application and built directly into ESX(i), and require no additional software installation. While the VCB Backup Framework still exists in vSphere and can also be used by backup applications, the vStorage APIs are the successor to VCB and will eventually completely replace it. The vStorage APIs are broken into four groups that have different types of functionality as listed below:

• vStorage APIs for array integration: Currently being developed with specific third-party storage vendors (i.e., EMC Corp., Hewlett-Packard [HP] Co. and NetApp), these APIs will allow vendors to leverage their storage array-based capabilities directly from vSphere. This includes things such as array-based snapshots, hardware offloaded storage device locking, integration between VMware and array-level thin provisioning, storage provisioning, replication, and more. This will enable vSphere to act more efficiently for some storage-related operations by allowing the storage array to perform certain operations.
• vStorage APIs for multi-pathing: These APIs enable third-party storage vendors to leverage array multipathing functionality through plug-ins that they can develop. These plug-ins allow for more intelligent storage multipathing to achieve better storage I/O throughput and storage path failover for a specific vendor storage array.
• vStorage APIs for Site Recovery Manager: These APIs are part ofVMware's Site Recovery Manager (SRM) and are used to integrate SRM with array-based replication for block and network-attached storage (NAS) models. This allows SRM to seamlessly handle both virtual machine, host failover and storage replication failover, and also enables SRM to control the underlying array-based replication that it relies on.
• vStorage APIs for Data Protection: These APIs are the ones that are very important to third-party backup and replication vendors as they enable better and more seamless integration to virtual machines disks. While designed to be the successor to the VCB, they include the functionality that was available in VCB and also added new functionality such as Changed Block Tracking (CBT) and the ability to directly interact with the contents of Virtual Disks via the VDDK.

The vStorage APIs are not really a single API and the term is basically just a name for a collection of interfaces that can be utilized by third-party applications to interact with storage devices in vSphere. These interfaces consist of various SDKs that exist in vSphere and also their Virtual Disk Development Kit (VDDK). The VDDK is a combination API and SDK that enables vendors to develop applications that create and access virtual disk storage. The VDDK is used in conjunction with other vStorage APIs to offer a complete integrated solution for management of storage in vSphere. For example, while VM snapshots can be managed using the SDK functionality, other operations like mounting virtual disks are handled through the VDDK.

Hot-add of virtual disks

The hot-add of virtual disks feature allows a virtual machine to mount the disk of another virtual machine while it is running so it can be backed up. This is similar to what was first introduced in VCB where a virtual disk can be mounted by another server to be backed up. The hot-add feature in vSphere allows one virtual machine running a backup application to mount the disk of another so it can read the data from it and write it to destination media. Doing this removes the backup traffic from the network as the VM running the backup application uses the VDDK to access the disk and all I/O requests to it are sent directly down the VMkernel I/O path.

The hot-add feature works by taking a snapshot of the virtual disk that deflects writes to a separate delta file. Once this is complete, the now read-only disk can be mounted by another VM so the data can be copied from it. Hot-add takes advantage of the SCSI specification that allows for SCSI devices to be added/removed from a server without powering it down. It works with disks on any type of storage supported by vSphere as long as the VM running the backup application is on a host that can access the storage of the target VM (i.e., shared storage). However, it does not work with VMs that have IDE virtual disks that are now supported in vSphere.

Several data backup applications have already taken advantage of the hot-add feature including VMware Data Recovery and Veeam Backup and Replication. The use of the hot-add feature is not available in all editions of vSphere and requires the more costly Advanced, Enterprise and Enterprise Plus editions.

iSCSI improvements

VMware made significant improvements to the iSCSI storage protocol in vSphere that resulted in increased performance and greater efficiency of virtual machines on iSCSI datastores. This is also beneficial to backup applications as the increased efficiencies with the iSCSI protocol are a direct benefit to heavy disk I/O operations that occur during virtual machine backups. The improvements to iSCSI in vSphere included the following:

• In vSphere, VMware made significant updates in iSCSI for both software and hardware initiators. The software initiator that is built into ESX was completely rewritten, tuned and optimized for virtualization I/O. The result of these efforts includes a marked improvement to performance as well as greater CPU efficiency which resulted in a significant CPU usage reduction when using software initiators.
• Support for Jumbo Frames was introduced in VI 3.5, but was not officially supported for use with storage protocols. With vSphere, VMware officially supports the use of Jumbo Frames with the iSCSI and NFS storage protocols. In addition, they now support 10 Gb Ethernet with iSCSI that results in much greater I/O throughput.
• Easier provisioning of iSCSI storage due to the iSCSI stack no longer requiring a Service Console connection to communicate with an iSCSI target. Configuration steps for iSCSI have been made easier and global configuration settings will now propagate down to all targets. Additionally bi-directional CHAP authentication is now supported for increased security.

These improvements make the use of iSCSI a more attractive choice over the more expensive Fibre Channel storage area network (SAN) for either virtual machine datastores or backup targets.

VMware Data Recovery

VMware introduced VMware Data Recovery (VDR) in vSphere that is a disk-to-disk backup application developed by VMware to provide basic backup capabilities natively in vSphere. VMware Data Recovery provides an alternative method for backing up virtual machines instead of the traditional OS agent methods that are used in physical environments. While not as feature rich as some of the other third-party backup applications, it does provide some advanced features such as inline data deduplication and compression, and a centralized management console that is integrated into the vSphere Client. In addition, VDR takes full advantage of the new features in vSphere such as Changed Block Tracking and hot-add of disk to ensure more efficient and faster backups. VDR is available as part of the Essentials Plus, Advanced, Enterprise and Enterprise Plus editions, or can be purchased a la carte with the Standard edition.

Veeam Backup and Replication software using the vStorage API for Backup and Replication.

Thanks,

Veera

Software Engineer,

Climb Inc,

http://www.climb.co.jp

Veeam Backup&Replication

http://www.veeam.com

Veeam Backup and Replciation for CBT and Thin Provisioning

Changed Block Tracking and vSphere backup

The vStorage APIs for Data Protection are most beneficial to backup and replication applications and vendors seem to be most excited about the new Changed Block Tracking feature that is included in it. This feature allows third-party applications to query the VMkernel to find out which disk blocks have changed in a virtual machines disk file since the last backup operation. Without this feature, applications would have to figure this out on their own which can be time-consuming. Now with CBT they can instantly find this out so they know exactly which disk blocks need to be backed up. This enables much faster incremental backups and also allows for near continuous data protection (CDP) when replicating virtual disk files to other locations. In addition, point-in-time restore operations are much quicker as CBT can tell exactly which disk blocks need to be restored to the virtual machine.

Changed Block Tracking is supported on any storage device and datastore in vSphere except for physical mode Raw Device Mappings, this includes iSCSI, VMFS, NFS and local disks. It also works with both thin and thick disk types. CBT is a new feature to vSphere, so it does require that the virtual machine hardware be version 7, which is the default in vSphere. By default, CBT is disabled as there is a very slight performance penalty that occurs when using it. It can be enabled on select VMs by adding parameters (ctkEnabled=true and scsi#:#.ctkEnabled=true) to the configuration file of the virtual machine, backups applications can also enable it using the SDKs. Once enabled, a VM must go through something called a stun/unstun cycle for it to take effect; this cycle happens during certain VM operations including power on/off, suspend/resume and create/delete snapshot. During this cycle, a VM's disk is re-opened, which allows a change tracking filter to be inserted into the storage stack for that VM.

The Changed Block Tracking feature stores information about changed blocks in a special "-ctk.vmdk" file that is created in each VM's home directory. This file is fixed length and does not grow and the size will vary based on the size of a virtual disk (approximately .5 MB per 10 GB of virtual disk size). Inside this file the state of each block is stored for tracking purposes using sequence numbers that can tell applications if a block has changed or not. One of these files will exist for each virtual disk that CBT is enabled on.

The vStorage APIs for Data Protection and the CBT feature make backups quicker and easier in vSphere and are a big improvement over VCB. VMware has provided third-party vendors with a much improved backup interface in vSphere, now it's up to them to adapt their products to take advantage of them.

Thin provisioning and backups

Thin provisioned disks are virtual disks that start small and grow as data is written to them. Unlike thick disks where are all space is allocated at the time of disk creation, when a thin disk is created its initial size is 1 MB, (or up to 8 MB depending on the default block size) and it then grows up to the maximum size that was defined when it was created as data is written to it by the guest OS. The benefit of thin provisioned disks is that they allow for the over-allocation of storage on a VMFS volume to make use of the often wasted unused space inside of a VM's disk. Thin provisioned disks are not new to vSphere and also existed in VI3, however, there were numerous changes to make them more usable in vSphere.

Why are thin disks important to backups? Many backup applications for virtualization do not operate inside the guest operating system and operate outside of it at the virtualization layer. Instead of backing up individual files inside the guest OS, they back up the single large virtual disk files (vmdk) that contain the encapsulated VM. Because of this, backup applications must search for empty disk blocks contained inside the virtual disk file so they do not back them up. This process of identifying empty blocks takes additional time and resources to complete. With thick disks all space is allocated at once, so a 40 GB virtual disk will actually take up 40 GB of disk space on a datastore regardless of how much space is used by the guest OS running on it. So, if only 10 GB of disk space is in actual use by the guest OS you will want to avoid backing up the extra 30 GB of empty space inside the virtual disk file.

Thin disks only take up as much space on a datastore as what is actually used by the guest OS, so if only 10 GB of a 40 GB virtual disk is in use the virtual disk file will only be 10 GB in size. Because of this, backup applications no longer have to worry about searching for those empty disk blocks because there are none in a thin disk. Not having to do this results in faster and more efficient backups which is just one of the advantages of using thin disks.

Veeam Backup provides the best solution for CBT and Thin provisioning in the Virtualization environment.

Thanks,

Veera

Software Engineer,

Climb Inc,

http://www.climb.co.jp

Veeam Backup and Replication

http://www.veeam.com

VirtualMachine Backup using Veeam

Veeam Backup and Replication :
---------------------------------
VM Backup process
--------------------
Prior to performing backup or replciation processes, it is necessary to add servers to add servers
you want to work with to veeam backup &replication. You may add ESX/ESXi server,vCenter and Linux servers.

To Add a Server, do one of the following:
Right click the servers node in the management tree and Select Add Server.
Click the Add server button in the tool bar.
Select the Add server command in the File menu.
Press Alt+A on the keyboard.

Adding ESX/ESXi server

--------------------------

Specify server type and name.

Enter a full DNS name or IP address of the server and select the server type ESX/ESXi host.

Specify server connection settings

You should enter administrator's credentials to connect to the ESX/ESXi server : Username and password.

Specify service console connection settings

This step is available if you are adding ESX server only.

Finish working with the wizard.
If you want to connect to the added ESX server on finishing work with the wizard, Select the connect when I click finish checkbox. Then click Finish button.

Creating a Backup job.

To perform Backup of a VM, you should create a backup job.

Before you begin.

prior to creating a backup job, make sure you have enough free space on the destination disk. When a backup job runs for the firsttime, full backup is performed. At all subsequent backups, only incremental data will be saved.

Launch a new backup wizard.

To run a new backup wizard, click Backup button in the toolbar. Alternatively, you can select Backup>Backup in the mainmenu.

Specify Job name and description.

At the first step of the wizard, enter the name and description of the job.

By default, the following description is initially provided for the created job.

You can select to perform backup in one of the three modes : VMWare storage API, VMWare consolidated Backup or Network Backup.

If the VMWare vStorage API mode is selected, choose the mode that should be used to transport VM data to the destination host: SAN with failover (recommented), SAN only and Network or virtual appliance.

Select VM to backup.

Click the Add button to browse to VMs that should be backed up. In the displayed VI tree, select the necessary object and Click the Add button.

Select the VM and click Add button.

Specify Backup destination,

From the destination list, select a host where the created backup should be stored.

Enable a VSS integration
Veeam Backup and Replciation allows you to create a transactionally consistent backup, ensuring successful recovery of vm applications without any data loss. To Enable this option, select the Enable veeam vss integration checkbox.

Select continue backup even if veeam vss quiescense fails if you want to continue backing up a virtual machine incase of any vss errors that may cause the operation to abort.

Define a job schedule, The job schedule step of the wizard allows you to choose to manually run the created job.

To specify the job schedule, select the Run the job automatically checkbox.

Finish working with the wizard. After you have specified a scheduled settings, click create . Select the Run the job when I click Finish checkbox if you want to start the created job
right after you complete working with the wizard. Click Finish button.

Confirm the job what job you are going to start.

Click Start option in the option list. Thats all. Now the backup job is running and storing the backup data into the destination folder.

Thanks,

Veera

Software Engineer,

ClimbInc,

http://www.climb.co.jp

Veeam Backup software

http://www.veeam.com

Changing Oracle DB Default ports

ORACLE DB Default port number change
-----------------------------------------
If you want to change the oracle port whether Tomcat using port number 8080.

You can simply execute the following statements in SQL *Plus.

The following section desctibes the syntax that can be used to alter XML DB configuration
information from within SQL*Pus. Ensure that you are logged into the database from a DBA account. In the following example, the default HTTP and FTP ports are changed from 8080 and 2100 to 8083 and 2111 respectively:

SQL> -- Change the HTTP port from 8080 to 8083
SQL> call dbms_xdb.cfg_update(updateXML(dbms_xdb.cfg_get(),'/xdbconfig/sysconfig/protocolconfig/httpconfig/http-port/text()',8083))
/
Call completed.

SQL> -- Change the FTP port from 2011 to 2111
SQL> call dbms_xdb.cfg_update(updateXML(dbms_xdb.cfg_get(),
'/xdbconfig/sysconfig/protocolconfig/ftpconfig/ftp-port/text()',2111))
/
Call completed.

SQL> COMMIT;
Commit complete.

SQL> EXEC dbms_xdb.cfg_refresh;
PL/SQL procedure successfully completed.

-----------Verify the change---------------
You can view the xml file whatever you changed in the CALL command.

SQL> set long 100000
SQL> set pagesize 9000
SQL> SELECT dbms_xdb.cfg_get FROM dual;

Thanks,
Veera
Software Engineer,
ClimbInc,
http://www.climb.co.jp

Database replication software
DBMoto http://www.hitsw.com

Virtualization software
Veeam http://www.veeam.com

SAP GUI Installation

SAP GUI INSTALLATION

Download the file sapgui.zip and place it in the folder you created. Once the zip file has been downloaded, double click on the zip file to extract the contents. A wizard will guide you through the extraction process. Make sure you "extract all" the files and choose a location you can find easily. After extracting the files, close the wizard window and open the folder where you saved the extracted files. Double click on theSapGuiSetup.exe to begin installation.

On the "Welcome Screen" please click NEXT.

The second screen is most important. You MUST click the blue text “Select All ". This will ensure all the necessary components of the GUI will be installed. After you have selected all the components, please click NEXT.

The third screen will let you choose a location to install the SAP GUI. You can either choose a custom location or accept the default by clicking NEXT.

When the installation finishes, click DONE to exit.

Regards,

Veera,

ClimbInc,

Product Support Engineer,

http://www.climb.co.jp

CGI/PERL Configuration in Apache Webserver

APACHE WEB SERVER CGI/PERL CONFIGURATION

1. Install the ApacheWebServer from Apache Site.

Publish Post

2. Install the Perl ActiveState If you are using Windows System.

Configuring Apache WebServer

1. Running Perl Scripts in a CGI directory

Edit httpd.conf file from C:\Program Files\Apache Group\Apache\conf

Find the line

ScriptAlias /cgi-bin/ "C:/Program Files/Apache Group/Apache/cgi-bin/".

It there is a hash mark(#) before the line, remove the hash character and enable it.

2. Running CGI Scripts anyWhere in your domain

1. If you don't want to be restricted to running CGI scripts within the ScriptAlias

directory in your domain, and want CGI scripts to run anywhere in your domain,

add the following line to your "httpd.conf" file.

AddHandler cgi-script .cgi

If you want the .pl extension recognised as a CGI script as well,

simply append the extension to the list, as follows:

AddHandler cgi-script .cgi .pl

Next, search for the line that says " in the file.

It should look something like this:

Options FollowSymLinks

AllowOverride None

Add "+ExecCGI" to the options list. The line now looks like this:

Options FollowSymLinks +ExecCGI

AllowOverride None

Modifying Your CGI Script

Your Perl script will typically contain an initial line of

#!/usr/bin/perl

This tells a Unix-based kernel to look for an interpreter at the path "/usr/bin/perl"

and invoke it to interpret the instructions in the file.

For example, on my Windows box,

I have to change the first line of my Perl scripts to:

#!c:/program files/perl/bin/perl.exe

Note : This is your perl installation directory.

Run the CGI/PERL Script from the Browser

Regards,

Veera

ClimbInc,

Product Support Engineer,

http://www.climb.co.jp

JBoss Installation and Configuration

JBoss Installation and Configuration in Windows system

-----------------------------------------------------------

1. Download JBOSS distribution files from the JBOSS site.

http://sourceforge.net/projects/jboss/files/JBoss/JBoss-6.0.0.M3

2. Set an Environment variable from MyComputer. Click Properties option.

Click Environment Variables button.

Click New button.

Add a Variable JAVA_HOME in the Environment wizard. Set the Java instllation folder.

Add a JBOSS_HOME variable and set the JBOSS installation folder.

3. StartUp

To Start JBOSS

Windows: Open an MS-DOS Prompt or Command Prompt window. cd %JBOSS_HOME%\bin run [-c default|minimal|all]

Note : minimal - This configuration includes only logging, the JNDI service, and the URL deployment scanner. You would want to use this configuration for starting services that don't require J2EE or as the base for a custom configuration. default - This configuration is the default. It includes all of the J2EE services exception RMI/IIOP and clustering all - This configuration includes all JBoss services.

If JBoss starts successfully, you should see something like this:

4. Open a JBOSS, to confirm the server is running.

5. If you are going to create a WAR file, you make sure the file structure. Folder -HTML/JSP Files -WEB-INF

	-

-lib -classes -web.xml

6. Create a WAR file and deploy into JBOSS application server. It is just copy a war file to deploy fodler.

7. Confirm wheather our application is deployed or not.

8. Open a browser and type the Application URL. http://localhost:8080/ClimbTest/CSVToJPEG.html

9. If we enter System.out.println line in the Program, it will be displaying in the server console.

10. To shutdown JBOSS server

Regards,

Veera,

ClimbInc,

Product Support Engineer,

http://www.climb.co.jp

Oracle Tuning Tips for Performance

These rules are simple but easy to follow and will yield lots of benefits:

Do not index every column in the table - have only those indexes, which are required.

Distribute your data to parrallelise the IO operations among multiple disks especially for those tables which have lots of IO's.

Try to avoid large data retrievals, since if the data being retrieved from a table is more than 10%-20% of the total data, Oracle will most likely do the FST.

Never do a calculation on an indexed column (e.g., WHERE salary*5 > :myvalue).

Whenever possible, use the UNION statement instead of OR conditions.

Avoid the use of NOT IN or HAVING in the WHERE clause. Instead, use the NOT EXISTS clause.

Always specify numeric values in numeric form and character values in character form (e.g., WHERE emp_number = 565, WHERE emp_name = 'Jones').

Avoid specifying NULL in an indexed column.

Avoid the LIKE parameter if = will suffice. Using any Oracle function will invalidate the index, causing a full-table scan.

Never mix data types in Oracle queries, as it will invalidate the index. If the column is numeric, remember not to use quotes (e.g., salary = 50000). For char index columns, always use single quotes (e.g., name = 'NAME').

Remember that Oracle's rule-based optimizer looks at the order of table names in the FROM clause to determine the driving table. Always make sure that the last table specified in the FROM clause is the table that will return the smallest number of rows. In other words, specify multiple tables with the largest result set table specified first in the FROM clause.

Avoid using subqueries when a JOIN will do the job.

Use the Oracle "decode" function to minimize the number of times a table has to be selected.

To turn off an index you do not want to use (only with a cost-based optimizer), concatenate a null string to the index column name (e.g., name||') or add zero to a numeric column name (e.g., salary+0). With the rule-based optimizer, this allows you to manually choose the most selective index to service your query.

If your query will return more than 20 percent of the rows in the table, use a full-table scan rather than an index scan.

Always use table aliases when referencing columns.

Best regards,

Veera,

Product Support Engineer,

ClimbInc,

http://www.climb.co.jp

DBMoto - MultiServer Synchronization

DBMoto 7 - MultiServer Synchronization setup

1. Start DBMoto Management Center from Start->Programs->HiT….

2. Create a New connection from Source folder.

3. Click Next button to continue.

4. Type the source name and choose the Database in the list box.

5. Specify the connection parameters for the source connection. Click Test button to confirm the connection parameters are correct.

6. For Transactional replication, to tick the checkbox and provide the sysadmin privilege user ID and password.

7. Select a Database tables you want to replicate. Click Next button to continue.

8. Click Finish button.

9. If you want to add additional tables in the source connection, Click the Select Tables… option.

10. Select a tables what you want in the list.

11. Choose the Targets option in the folder and click Add New Connection…

12. Click Next button to Continue.

13. Type the target connection name and specify the Database name. Click Next button to continue.

14. Type the connection parameters for the target connection. Click Next button to continue.

15. Check the option and click the Verify button. Click Next button to continue.

16. Click OK button to upgrade DBMotoLIB.

17. Click Install button.

18. Click Yes button.

19. Click Next button to continue.

20. Select a tables what you are going to replicate. If you want to create new tables, just leave it blank. Click Next button to continue.

21. Click Finish button.

22. Follow the screen to create a target table.

23. Click Next button to continue.

24. Select a source table to be copied. Click Next button to continue.

25. Select the target connection and define the target table name.

Click Next button to continue.

26. Click Next button if you don’t want to change datatype.

27. Click Next button to continue.

28. Click Finish button.

29. Click Yes to add the created tables in the target table list.

30. Similarly create another two connection and select a tables.

31. Create Multiple Replications… option in the source connection list.

32. Click Next button to continue.

33. Select Synchronization in the Replication Mode. Click Next button to continue.

34. Select the source connection and library/database/schema to be replicated. Click Next button to continue.

35. Set the correct transaction ID to read transaction from SQLServer.

36. Click Next button to continue.

37. Select Connection Name and Library Name. Click Next button to continue.

38. Set the correct journal/receiver from the iSeries Server.

39. Click Next button to continue.

40. Select tables to replicate and Click Next to continue.

41. Click Next button to continue.

42. Click Start button to continue.

43. Click Close button.

44. Specify the Replication Mode as Synchronization, Click Next button to continue.

45. Specify the source connection database what you are going to replicate. Click Next button to continue.

46. Click the read button to set the transaction ID. Click OK button in the dialog box.

47. Specify the target connection to be replicated. Click Next button to continue.

48. Click Read button to set the transaction ID. Click OK button.

49. Click Next button to continue.

50. Specify the tables to put into replication. Click Next button to continue.

51. Click Next button to continue.

52. Click Start button to create replications.

53. Click Close button to continue.

54. Create a New Multiple replication for Synchronization. Click Next button to continue.

55. Select a Source connection to be replicated. Click Next button to continue.

56. Click Read button to set the transaction ID. Click OK button to close the dialog box.

57. Click Next button to continue.

58. Select a target connection to be replicated. Click Next to continue.

59. To set the transaction ID from Oracle redo log. Click Next button to continue.

60. Click Next button to continue.

61. Select tables what you want to replicate. Click Next button to continue.

62. Click Next button to continue.

63. Click Start button to continue.

64. Click Close button.

65. Create a New group for MultiServer Replication.

66. Type the Group Name and click Next button.

67. Select a replication to insert into a group. Click Next button to continue.

68. Click Finish button to complete the wizard.

69. Click Yes button.

70. Start a DBReplicator from System tray or from menu.

71. Confirm the replication process in the monitor.

--------------------------------------The End-------------------------------------------

Regards,

Veera,

Product Support Engineer,

http://www.climb.co.jp

----------------------------------

For More Information

http://www.hitsw.com/products_services/dbmoto.html