Thursday, June 30, 2016

Unable to Protect VM with Snapshots in VMware SRM

Setup


VMware vCenter 5.5 Update 3a
SRM 5.5.1
ESXi 5.1 Update3

Issue


When trying to Configure Protection for the VM it goes till 99% after getting an error. The error details

Internal error: Received unexpected exception from multi-site operation.

Analysis


We tried usual steps such as

  • Restarting SRM Service/Server
  • Ensure it is not licensing issue
  • VM Hardware version upgrade from 8 to 9


Upon log analysis we found Customer had 90 snapshots for the VM which was for SQL. Surprisingly it didn’t appear under normal place of Right Click Manage Snapshots however when we check datastore folder it appeared there


Resolution


As per VMWare ESXi only supports 32 levels of snapshot on a VM however VM in question had 93 levels of snapshots, and this is making it fail to be protected on SRM.  We were able to protect VM once snapshots were consolidated



VMware SRM Recompute Device Groups Occurence

Setup


VMware vCenter 6.0 Update 1b
SRM 6.1
NetApp SRA 2.1P1

Issue


We have upgraded DR vCenter Server to 6.0 U1b and DR SRM Server to 6.1. After upgrading and configuring SRM we observed that Production vCenter keeps generating Re-compute Device Groups tasks every 30 seconds

Analysis


The re-computes are triggered by the SRA. Under normal circumstances Datastore Group computation is triggered by the following events:

  • Existing VM is deleted or unregistered
  • VM is storage vmotioned to a different datastore
  • New disk is attached to VM on a datastore previously not used by the VM
  • New datastore is created
  • Existing datastore is expanded


In our case none of this was happening, there are APD for other datastores in PROD vCenter however it is not included in the SRM Configuration. SRM is configured with single datastore in Array Managers Volume include list

I am wondering if anyone else has come across similar issue like this

VMware SRM Fault

Setup


VMware vCenter 5.5 Update 3
SRM 5..8.1

Issue


Protection Group summary page shows 25 VMs as not configured. When checked in the in protection group, VMs with warnings are not getting displayed.

Analysis


Upon troubleshooting with VMware Support, we got notified that they suspect this might be a defect in the software that we will have to escalate to engineering. Engineering confirmed this to be fault and working on developing hot patch

I am wondering if anyone else has come across similar issue like this


Tuesday, June 28, 2016

VMware SRM Test Failover/ Planned Migration Fails with Error - There are not enough licenses installed to perform the operation (2145522)

Symptoms
· Running a Test Recovery from Protected and/or Recovery site fails
· Running a Plan Migration  from Protected and/or Recovery site fails
· Even after adding the license to the vCenter Server, you see error:
There are not enough licenses installed to perform the operation.
Cause
This issue occurs when a license is not assigned to an individual asset. 
If you have re-installed vCenter Server or never used the SRM from Recovery to Production site, the license for an SRM asset can be unassigned. 
Resolution
To resolve this, ensure the key is assigned to the SRM solution in the vCenter Server for re protecting the virtual machines on both production and disaster recovery site. You have to login to each vCenter separately and check.


The procedure to assign the license if not available is:

For 5.x C# Client:
1. In the vSphere Client, select Home > Administration > Licensing.
2. Click Manage vSphere Licenses.
3.  Click Next to navigate to the Assign Licenses page.
4. Click the ESX, vCenter Server, or Solutions tab to display the available assets.
5.  Select the assets to show.
6.  In the Asset window, select one or more assets to license.
7. To select multiple assets, use Ctrl-click or Shift-click.
8.  In the Product window, select an appropriate license key and click Next.


For 6.x Web Client:
1.  In the vSphere Client, select Home > Administration Licensing.
2. Click the Assets tab.
3. Click Solutions session.
4. Identify and select the SRM extension.
5.  In the All Actions Menu, select Assign License.
6. In the Assign License window, select an appropriate license key and click Next.    

If the license key you assign has a strong limit, the license capacity must be greater than or equal to the required license use for the asset. Otherwise, you cannot assign the license key. Check the EULA of the license to determine whether it has a strong limit.

Note: If you are Reprotecting your virtual machined, ensure that this has been done for both Protected and Recovery vCenter Servers.


Friday, June 3, 2016

Microsoft Convenience Update and VMware VMXNet3 Incompatibilities

Microsoft recently released a “Convenience Update” patch for Windows 7 and Windows Server 2008 R2 SP1. This update has incompatibility issues with virtual machines running on the VMware vSphere virtualization platform. This incompatibility is confined to one specific configuration scenario – It impacts VMs that use the VMware VMXNet3 virtual network adapter type

Here is the incompatibility issue as described in Microsoft’s announcement of the Update:


Known issue 1


Symptoms


A new Ethernet vNIC may be created with default settings in place of the previously existing vNIC, causing network issues.  Any custom settings on the previous vNIC are still persisted in the registry but unused.

Resolution


To resolve this issue, uninstall the convenience rollup.

Status


Microsoft is investigating this issue to determine proper course of action with VMWare. To resolve this issue uninstall the convenience rollup. Further information will be posted here as the investigation continues.



Thursday, May 19, 2016

Goodbye vSphere Client for Windows (C#)

VMware has announced C# client (AKA Desktop Client/thick client/vSphere Client for Windows) will not be available for the next version of vSphere. .  It will be replaced with the HTML 5 Web Client which is a VMware Fling today

Current versions of vSphere (6.0, 5.5) will not be affected, as those will follow the standard support period


Friday, April 29, 2016

Migrate Port Groups from Standard Switch to Distributed Switch

In our current environment we use to create port groups on vSphere Standard Switches. It was fine until the environment was small.

Once it reached 5 Hosts and about 25 odd Port Group we started observing issues of inconsistency and issues being observed in terms of naming conventions, VLAN id not entered properly , port groups not created on all hosts and so on

We decided to migrate the Virtual Machine Port Groups to Distributed Switches. There might be multiple ways of doing it however following we felt is the best way forward

  • Get the current Port Groups of the vSphere standard switches and export CSV using following command
Get-VirtualPortGroup -VirtualSwitch vSwitch0 | Select Name,VLanID > C:\vSSPortGroups.csv
  • Create Distributed Switch using WebClient 
  • Work with Network Team to ensure VLANs are passed on the uplinks of the Distributed Switch. In our case we had trunk all so we were covered
  • Create Port Groups in Distributed Switch using following command. We preferred to do it manually by repeating the command individually for all Port Groups. We could automate using CSV if required  
Get-VDSwitch -Name "DVSwitch Name" | New-VDPortgroup -Name "DVSwitch Port Group name " -NumPorts 8 -VLanId xxx
  • Migrate Virtual Machine Networking to the vSphere Distributed Switch using following process. We preferred to do it manually for each port group so that we could validate and ensure we are not losing connectivity. I know we can expedite but it depends on lot of things ( Non-Technical obviously J )
http://pubs.vmware.com/vsphere-60/index.jsp#com.vmware.vsphere.networking.doc/GUID-0E91679E-A896-45B9-A489-5325BE1E86BB.html
  • Once the migration is completed , Remove the Port Groups on vSphere Standard Switches using following command
Get-VMHost | Get-VirtualSwitch -Name vSwitch0 | Get-VirtualPortGroup -Name "vSS Port Group Name " | Remove-VirtualPortGroup -confirm:$false



Please feel free to add in your comments

Tuesday, March 8, 2016

VMware vCenter Site Recovery Manager service fails to start

Setup


VMware vCenter 5.5 Update 2e
SRM 5.5.1

Issue


SRM Sites Unable to Connect

Analysis


Upon troubleshooting, we have noticed that customer has upgraded their vCenter server from 5.5 U2e to 5.5 U3b.
In vCenter 5.5 U3b, SSLv3 is disabled by default due to POODLE vulnerability. Due to this upgrade SRM (version 5.5) lost SSL connectivity to vCenter server which meant SRM service will not start



Resolution


To fix this issue, we need to upgrade SRM from 5.5 to 5.8.1



Simplified VMware Certification Program Exam Registration Process

As of March 5th 2016 VMware and Pearson VUE have made system changes to simplify the certification exam registration process. You will now find that exam registration is a seamless experience when you log into your VMware MyLearn account.

Just select the “Register Now” button or link while navigating through the VMware Certification website and you will be directed into your Pearson VUE\VMware profile to complete your exam registration.

No more exam authorization required! No more multiple accounts or log-ins between Professional and Advanced registration necessary. Just ONE click after you have logged into you VMware MyLearn account



Friday, March 4, 2016

VMware SRM 6 Demo


Recently we had to record video and one of my collegeau did an excellent job of recording the SRM 6.0 Demo

This video includes following

  1. Test Recovery and Clean up to simulate Test 
  2. Planned Migration to DR simulating scheduled maintenance and Planned Migration back to PROD once maintenance is completed 
  3. Disaster Recovery in event to a True Disaster when Production Site is down Completely


Site Recovery Manager 6.0 Configuration Continued

This is Part 2 of Configuration of VMware Site Recovery Manager 6.0

Adding Array Manager and Enable array pair


In this Step we will be adding Storage appliance IP address as per their respective sites.

Go to Sites, click on one of the sites,

Under Summary, click on Add array manager and enable array pair.


Select Add a pair of array.

Click on Next.



Select the Site.

Click on next.


Select the SRA from the List, if we have 2 or more SRA installed on the SRM server. In our case we just have 1

Click on Next.


Give a name to the array manager in Display name.

Enter EMC RPA Management IP.

Admin Username and password.

Click on Next.


Now give the other Site information below

Give a name to the array manager in Display name.

Enter EMC RPA Management IP.

Admin Username and password.

Click on Next.


Check the box to enable the array pair.

Click on Next.


Check all the information under ready to complete.

Click on Finish.


Go to Sites and select array based replication.


On Selecting the array manager, under manage you should be able to see the array pair and the volumes replicating from Prod to DR.

Creating Protection Group


In this Step we will be Creating Protection Groups

Go to Sites, click on one of the sites,

Under Summary, click on Create protection group


Give a name to the Protection Group.
Select the Sites
Click on next.


Under Protected Site select the DR site.

In Replication type Select array based replication.

Under array pair select the Recover Point.

Click on Next.


Here you will find the list of all replicated group of datastore.

If you want to create only one protection group select all at once or you can create one protection group for each datastore groups getting replicated.

Select the check box and click on next.


Click on the Finish.


Now under Protection group à related objects à Virtual machine tab you should be able to see all the protected VM and its status.


Creating Recovery Plan


In this step we configure a recovery plan according to which VM gets recovered at DR at the time of Disaster or test.

Go to Sites, click on one of the sites,

Under Summary, click on Create a recovery plan.



Give a name to the Recovery Plan.

Select the Sites

Click on next.


Select the recovery Site.

Click on Next.


Select the Protection Group from the List to be recovered. It can be one or more depending on the requirement.

Click on Next.



Select the networks from the drop down under Test network.

Click on Next.



Check the Ready to complete details

Click on Finish.


Now SRM is fully configured and ready to execute Test Failover / Disaster Recovery. Always perform Test failover after initial configuration so ensure it is working as expected

Site Recovery Manager 6.0 Configuration

This is Part 1 of Configuration of VMware Site Recovery Manager 6.0

Granting access for pairing the sites


Below Steps for granting access to Administrator Group should be followed on both the ends i.e. it should be added on Production and DR.

Login to vCenter web Client.



Go to Administrator on Home Page.


In Users and Groups, go to Groups Tab.
Select Administrator Group. Under Group Members we need to add the service account created by Customer for getting Administrator access on SSO and vCenter.



Search for the user, select the user and click on add to complete the step.


Under Group members you should be able to see the added user.



Site Pairing


Login to vCenter web Client.

Go to Site Recovery.


In Sites right click on SRM server and then click on Pair Site.



As we are pairing the sites by login to DR vCenter, give Production PSC address (it is embedded in this case so give production vCenter IP address).

Click on Next.




Put down the credentials having access to PSC server.

Click on Finish.


After finishing the wizard you will get 4 different warning popup one by one for certificates, click on Yes for all of them.

                       


  

                   

As soon as Site pairing is completed you will be asked to Login to the Site. Enter the credentials to complete the step.


Now you should be able to see both the sites on Summary.


Creating Resource Mapping


In this Step we will be mapping various resources (Cluster, Resource pool) of Production and DR Sites.

Go to Sites, click on one of the sites,

Under Summary, click on Create Resource mapping.



Select the Cluster and resource pool to be mapped between Production and DR.

Click on Add Mapping.

Click on Next.



If reverse mapping has to be configured select the mapping.

Click on Finish.


Creating Folder Mapping


In this Step we will be mapping Folder structure (Folders under VM and Templates View) of Production and DR Sites.

Go to Sites, click on one of the sites,
Under Summary, click on Create folder mapping.


Select Prepare mapping manually.

Click on Next.




Select the folder to be mapped between Production and DR.

Click on Add Mapping.

Click on Next.




If reverse mapping has to be configured select the mapping.

Click on Finish.


Creating Network Mapping


In this Step we will be mapping Network Port Group of Production and DR Sites.

Go to Sites, click on one of the sites,

Under Summary, click on Create Network mapping.


Select Prepare mapping manually.

Click on Next.



Select the Port Groups to be mapped between Production and DR.

Click on Add Mapping.

Click on Next.



If reverse mapping has to be configured select the mapping.

Click on Finish.


Configuring Placeholder Datastore


In this Step we will be Configuring Placeholder datastors on the sites. This step has to be repeated on both the sites individually.

Go to Sites, click on one of the sites,

Under Summary, click on Configure placeholder datastore.


Select the datastore.

Click on ok.


Next Part will include 

Configuring and Enabling Array Manager. 

Creating Protection Group and Recovery Plan