Tag / exchange 2013

by judeperera

Exchange ECP Empty Results Issue in Internet Explorer

When it comes to browser compatibility issues, it’s always about non-Microsoft browsers having issues with Exchange Servers. But in this particular case, the issue was vise-versa.

This was an Exchange 2013 environment, and the administrator reported that while using the ECP in Internet Explorer he was unable to view any data within sub windows and the links within were not clickable. However, when using Google Chrome or FireFox, this was not a problem.

The above image is what you get when you open a Recipient mailbox in ECP. As you can see, the values are all empty and I cannot click any of the action items on left side.

Now let’s see what type of an approach you should take in a scenario like this.

  • Check with multiple Internet Explorer versions/users. Plus clear your cache and cookies as well.
  • Check with compatibility mode on IE.
  • Check if the issue is for both internal and external users. Why I’m saying this is that in most of the cases your Exchange Servers may be placed behind many layers such as proxy, load balancers etc. So, we need to narrow this down to internal or external.
  • Check if the issue is for a single server only. Go to your exchange server, open Internet Explorer and check your ECP using localhost FQDN. By this, we can see if the issue is only for one Exchange Server or multiple servers.
  • Now that you have figured out where and which servers are affected, it’s time to refresh your IIS cache. We do this because that’s the first point where all web connections hit to. And I’ve seen that in most client connectivity issues, doing a simple refresh or may be a reset on IIS fixes the issue. Resetting IIS would be the last resort as it would drop all connections and users will not be able to connect until the service starts which can take some time.

Below are the steps that worked for me;

  1. Open IIS Manager and navigate to Application
    Pools.
  2. Under the list of pools, you can see the MSExchangeECPAppPool. This is responsible for ECP connections and we can easily refresh the cache.
  3. Right click the MSExchangeECPAppPool and click Recycle.

  4. Now IIS will refresh your specific app pool.
  5. Login back to the ECP and check if the issue is sorted.
  6. In my case, it was working again.

Advertisement
by judeperera

[Script] Shared Mailbox Convert and Enabling Sent Items Copy

This is a simple script that will allow an administrator to achieve the below tasks;

  1. Convert a User mailbox to a Shared Mailbox
  2. Enable Messages sent from the shared mailbox to be saved to the Sent Items folder of the shared mailbox

Let’s take the below scenario;

User Mailbox – user@contoso.com
Shared Mailbox – shared@contoso.com (user@contoso.com has SendAs and FullAccess permissions)

In a case where if user(user@contoso.com) tries to send an email with SendAs/SendOnBehalfOf permissions for ‘shared@contoso.com‘, the sent email will only be available in the primary users ‘Sent Items‘ ONLY. In a business requirement where the mail should also be present in the ‘shared@contoso.com‘ shared mailbox, we would require the below permissions to be enabled. Once done, the email with SendAs/SendOnBehalfOf permission will be in both user@contoso.com and shared@contoso.com mailboxes’ Sent Items.

The MessageCopyForSendOnBehalfEnabled parameter specifies whether to copy the sender for messages that are sent from a mailbox by users that have the “send on behalf of” permission. In this script we will be enabling this by setting the value to $true; where when a user sends a message from the mailbox by using the “send on behalf of” permission, a copy of the message is sent to the sender’s mailbox.

The MessageCopyForSentAsEnabled parameter specifies whether to copy the sender for messages that are sent from a mailbox by users that have the “send as” permission. In this script we will be enabling this by setting the value to $true; where when a user sends a message from the mailbox by using the “send as” permission, a copy of the message is sent to the sender’s mailbox.

Want to give a try? you can get the script here..


 

by judeperera

Restore a Deleted User Mailbox

Issue

In this scenario we are going to take an on-premise environment with a Windows Server 2012 R2 Active Directory and an Exchange Server 2016 environment. In our case, a user was accidentally deleted from the OU itself. (NOTE: It is always a best practice to enable ‘Protect object from accidental deletion‘ for your AD Objects.

You can also recover a deleted mailbox using backups. However, note that backups are not current. This will lead to restoring but will not give you the latest data once recovered.

Prerequisites

  1. First, we need to get the AD account restored. For this, you should have the Active Directory Recycle-bin enabled in your environment. If you don’t have this enabled already, you are in tough luck.
  2. Secondly, we need to make sure that you are within the time duration mentioned in the ‘Keep deleted mailboxes for (days)’ under Exchange database properties where the user was. In our case, it was 30 days as shown below;

Solution

  1. Restore the AD account from Active Directory
  2. Ensure that restored user accounts ‘User logon name‘ has the correct domain mentioned.
  3. Go to AD User properties page, note down the ‘Display Name‘ of the user (ex: Jude Perera).
  4. Open Exchange Control Panel (ECP), go to Recipients > Mailboxes. Click More
    Options icon, and then click Connect a mailbox

  5. Click the deleted mailbox that you want to connect.
  6. Note down the ‘Display Name‘ of the user you want to restore (ex: Jude Perera).
  7. Open the Exchange PowerShell and run the below command;
    Connect-Mailbox -Identity "Jude Perera" -Database DB01 -User "Jude Perera" -Alias jude

    NOTE: The Identity parameter specifies the display name of the deleted mailbox retained in the mailbox database provided. The User parameter specifies the Active Directory user account to connect the mailbox to. Alias is what the users email alias is.

    Now we must get the identity and user attributes from the noted in steps 3 and 6 which is as below;

    Connect-Mailbox -Identity "Jude Perera" -Database DB01 -User "Jude Perera" -Alias jude

  8. Once you run the above command the mailbox will now be connected.
  9. To verify, go to the Exchange Admin Center and search for the user mailbox in the ‘Recipients‘. Your reconnected user will be shown.

Tip: In case you receive an error ‘Property expression “jude” isn’t valid. Valid values are: Strings that includes ‘@’, where ‘@’ cannot be the last character.’ Navigate to the AD user properties, select the ‘Account’ tab and verify that the User logon name has the domain selected properly.

by judeperera

Move Transport Database Files in Exchange 2013/2016 Step by Step

Issue:

In terms of sizing an Exchange Server environment, it is always advised and recommended to follow the Microsoft Sizing guides, this specifically means that you need to go through the Exchange Sizing Calculator. The tool gives you estimated values of how your databases, logs and transport queues are going to grow. So, if you do not properly plan these sizes, you might end up getting your disks full.

The scenario which I’m going to talk about is such situation where the free space of the C Drive almost got full. While digging in what’s being eating up the storage, we found out that the Exchange Transport queue, or Mail.que file is the culprit.

Exchange Transport Queue

A queue is a temporary holding location for messages that are waiting to enter the next stage of processing or delivery to a destination. Each queue represents a logical set of messages that the Exchange server processes in a specific order. In Exchange 2016, queues hold messages before, during and after delivery. Queues exist in the Transport service on Mailbox servers and on Edge Transport servers.

File Description
Mail.que This queue database file stores all the queued messages.
Tmp.edb This temporary database file is used to verify the queue database schema on startup.
Trn*.log Transaction logs record all changes to the queue database. Changes to the database are first written to the transaction log and then committed to the database. Trn.log is the current active transaction log file. Trntmp.log is the next provisioned transaction log file that’s created in advance. If the existing Trn.log transaction log file reaches its maximum size, Trn.log is renamed to Trn nnnn.log, where nnnn is a sequence number. Trntmp.log is then renamed Trn.log and becomes the current active transaction log file.
Trn.chk This checkpoint file tracks the transaction log entries that have been committed to the database. This file is always in the same location as the mail.que file.
Trnres00001.jrs
Trnres00002.jrs		 These reserve transaction log files act as placeholders. They’re only used when the hard disk that contains the transaction log runs out of space to stop the queue database cleanly.

Solution:

In simple, we can move the Mail.que database and log files associated to a different location. The below step by step guide will take you through how you can achieve this.

Before going ahead, here are some tips when it comes to moving your queue database.

  • Ensure that the destination disk/drive has enough and additional buffer space, remember that during peak times, this can grow. If it’s possible to attach a separate disk for this, go ahead. It’s even better.
  • The move process requires the Exchange Transport service to be stopped until the data is moved to the new location. This means that there will be a downtime where mail flow on the server will be interrupted.
  • The transport queue files are located in the below path 
    %ExchangeInstallPath%TransportRoles\data\Queue

Once you have the disk and the downtime planned, we can start the procedure.

  1. Navigate to the location that you would will be moving the data to.
  2. Create a folder where the queue database and transaction logs will be moved. In my case, I’m moving the data do the below path;
    "F:\Program Files\Microsoft\Exchange Server\V15\TransportRoles\data\Queue"
  3. Right click on the above folder “Queue”, select Properties
  4. Navigate to Security tab, click on Edit under Change permissions

  5. Under Permissions verify that the below accounts are listed and the shown permission level is present. It not, add the user/service account and assign permissions
    1. Network Service: Full Control
    2. System: Full Control
    3. Administrators: Full Control
  6. Click OK to apply the permissions to the folder.
  7. Open Notepad using Run as Administrator
  8. Using notepad, click Open and Navigate to the below path

    %ExchangeInstallPath%Bin\

  9. Open the EdgeTransport.exe.config file (you may want to take a backup of the file in case something goes wrong)
  10. On the config file lookup for the following content;
    <add key="QueueDatabasePath" value="<CurrentLocation>" />
<add key="QueueDatabaseLoggingPath" value="<CurrentLocation>" />
  11. Now we need to modify the <CurrentLocation> and replace it with the new path for the queue files. In our case this will be as below;
    <add key="QueueDatabasePath" value="F:\Program Files\Microsoft\Exchange Server\V15\TransportRoles\data\Queue" />
<add key="QueueDatabaseLoggingPath" value="F:\Program Files\Microsoft\Exchange Server\V15\TransportRoles\data\Queue" />

  12. Save and close notepad.
  13. Open Services.msc
  14. Stop the Microsoft Exchange Transport Service.

  15. Navigate to the below path where the old queue files are located at;
    %ExchangeInstallPath%TransportRoles\data\Queue
  16. Take a backup of all the files in the folder into a different location just in case.
  17. Move existing database files (Mail.que, Trn.chk, Trn.log, Trntmp.log, Trn nnnnn.log, Trnres00001.jrs, Trnres00002.jrs, and Temp.edb) to the new location. This is the location you mentioned in Step 11.

  18. Go to Services.msc and Start the Microsoft Exchange Transport service.

  19. Monitor the status of the new location and the files.
  20. Verify that the old path is empty and no new files are being created.
  21. Send a few mails with attachments to verify and monitor mail flow.

by judeperera

450 4.7.320 Certificate validation failed

Issue

The environment was running Exchange Server 2016 (n-1)CU with Office 365 in a hybrid mode. The issue came up when users on-prem noticed that they are unable to receive emails from their own O365 cloud users. Looking more into this, figured out the following user mail flow scenarios;

  • On-prem users can send/receive mails within on-prem and to/from internet users
  • O365 cloud users can send/receive mails within O365 and to/from internet users
  • On-prem users cannot send/receive emails to/from O365 users

So obviously, this had to do something with the hybrid connector since that’s the tunnel that bridges the two environments.

So back to the basics, lets trace down the error messages and see if we can get a clue. In order to see where our mails are stuck, you need to first check the delivery reports. Log in to the O365 Exchange ECP as administrator and navigate to Message Tracing. On the message tracing log, you will see something like below;

11

We’re almost there now. The above indicates that why O365 is not delivering the mails to on-prem environment is because as on the first item, we have configured to use TLS between the hybrid environments. Thus due to whatever reason, the certificate presented from the on-prem public IP does not match the certificate that is been binded in the Hybrid Configuration. Now in order to validate this, we need to look at two points;

  • Ensure that the certificate is not expired
  • Check Hybrid Configuration certificate settings: To validate that you have the correct certificate open up the Exchange PowerShell module on your on-prem and run the below command;

Get-HybridConfiguration

12

Now as you can see, the Hybrid configuration is configured with the correct certificate. So nothing to worry on that.

  • Exchange Connector TLS settings: The next connection point would be our internal Exchange Server receive connector. Why this connector is important is that when O365 is trying to connect with the exchange server for mail delivery, it will try out TLS for authentication. This is because we have TLS enabled; meaning we should have the correct certificate binded to the corresponding Receive Connector. In many cases this would be the Default FrontEnd connector. Run the below command and ensure that the property tlscertificatename is set correctly and is the same as the above certificate.

Get-ReceiveConnector -Identity “<ExchangeServerName>\Default Frontend <ExchangeServerName>” | fl

In my case, the certificate was all fine and is set for the one as expected.

So, the certificate itself, exchange setup and office 365 hybrid setup is all configured as expected. Yet somehow, Office 365 is telling me that the certificate is not correct.

This is where we would need to properly test TLS in a more informative manner. We will now see how we can use a method where we can see the TLS communication that is made to our on-premises.

Enter CheckTLS web testing provider. The provider has many tools to check TLS mechanisms but in our case, we are looking at the receiving part. Thus we will head over here.

https://www.checktls.com/TestReceiver

What it does?

TestReceiver performs all the steps that Internet email systems go through to send email. It records every command and byte of data it sends and every answer and byte of data that the other email system sends. TestReceiver never actually sends an email, it just gets as close as possible, learning as much about the remote system as it can.

Because CheckTLS focuses on security, TestReceiver tries to establish a secure (TLS) connection with the recipient’s system. Along with recording everything, it looks at the security of the recipient’s system for things like: certificate contents and signers, encryption algorithms, key lengths, hostname mis-matches, incorrect wild-card usage, weak cyphers, etc.

So is this safe? absolutely. Since it only checks whatever is already published by your organization. It doesn’t grab your email ID, username or passwords.

  1. So head over to the site and under the Input fields, type your domain name in the eMail Target
  2. Ensure that under the Output Format you have selected Detail (this will provide you a verbose log of the connection status, more meaningful)
  3. Click Run Test.
  4. Give it some time to run the test and notice the logging which happen in the background. This is the session initiation that happens in real time.
  5. Once the test is completed, look in the detailed log.13

 

And we have found the culprit. Just have a closer look at the above image. You can see that the TLS authentication does start however the certificate validation fails. The reason is that the issuing certificate is not our Exchange Certificate, but a self signed certificate by the barracuda appliance which is front-ending to external connections.

When Office 365 tries to initiate a TLS session, it is getting this self signed certificate thus the required URLs are not found (mail.domain.com) hence it drops the connection and throws out;

450 4.7.320 Certificate validation failed.

If this is your case, bump up your network team and ask them to bind the correct certificate from the appliance. Once its configured properly, run the below tests again to validate that everything is all good.

  • CheckTLS Receive validation – Ensure correct certificate is thrown out
  • Office 365 Connector Validation

If everything is successful, send out couple of emails and you will see that mailflow is working as expected. For the mails that were queued, give some time and it’ll start flowing.

Got any inputs? Please feel free to let me know your ideas.

Cheers..

by judeperera

Exchange Server Upgrade | Step-by-Step

The guide will cover upgrading an Exchange Server 2013 CU8 into Exchange 2013 CU15.

In a nutshell, this is the procedure:

  1. Prepare the existing environment
    1. Download the latest binaries
    2. Remove Interim Updates
  2. Perform Upgrade
    1. Enable Exchange Server maintenance mode (High Availability)
    2. Upgrade Schema
    3. Install Exchange Binaries
    4. Enable Exchange Server services

Prepare the existing environment

Download Exchange Server SP/RU/CU binaries

When you are performing an Upgrade, you can go ahead and install the latest Cumulative Update (CU). You DO NOT, need to install all previous CU’s one by one in an incremental way. This is because a CU is a full installation of Exchange Server plus a collection of all the updates, patches and changes that has been made available so far.

For an Example, if you are on Exchange Server 2016 RTM, you can straight away install CU4. Because CU4 will contain all changes made in each of the CU’s previously released.

Exchange 2016

Exchange 2013

Version Blog post
Exchange 2013 CU15 Released: December 2016 Quarterly Exchange Updates
Exchange 2013 CU14 Released: September 2016 Quarterly Exchange Updates
Exchange 2013 CU13 Released: June 2016 Quarterly Exchange Updates
Exchange 2013 CU12 Released: March 2016 Quarterly Exchange Updates
Exchange 2013 CU11 Released: December 2015 Quarterly Exchange Updates
Exchange 2013 CU10 Released: September 2015 Quarterly Exchange Updates
Exchange 2013 CU9 Released: June 2015 Exchange Cumulative Update and Update Rollups
Exchange 2013 CU8 Released: Exchange Server 2013 Cumulative Update 8
Exchange 2013 CU7 Released: Exchange Server 2013 Cumulative Update 7
Exchange 2013 CU6 Released: Exchange Server 2013 Cumulative Update 6
Exchange 2013 CU5 Released: Exchange Server 2013 Cumulative Update 5
Exchange 2013 SP1 Released: Exchange Server 2013 Service Pack 1
Exchange 2013 CU3 Released: Exchange Server 2013 Cumulative Update 3
Exchange 2013 CU2 Released: Exchange Server 2013 Cumulative Update 2
Exchange 2013 CU1 Released: Exchange Server 2013 Cumulative Update 1
Exchange 2013 RTM Exchange Server 2013 Reaches General Availability

Remove Interim Updates

Now that you have downloaded the binaries, you need to get rid of the Interim Updates that may have been installed in your environment. In some cases, if an Interim Update is installed, Microsoft Exchange Server CU’s or SP’s cannot be installed. Therefore, before installing the binaries, read the release notes for any information on removal of interim updates. Below steps will guide for that;

  1. In Control Panel, double-click Programs and Features.
  2. In the Currently installed programs list, click Interim Update for Exchange Server 201X (KBxxxxxx), where xxxxxx is the Knowledge Base article number that is associated with the IU.
  3. Click Remove.
  4. At a command prompt, run sn.exe -Vu * to enable strong name verification.
  5. Run sn.exe –Vl to verify that strong name verification is enabled.

Perform Upgrade

Enable Exchange Server maintenance mode (High Availability)

If you have standalone servers in your environment, then an upgrade will require you to go in for a downtime. Why? Because the services needs to be stopped during the upgrade process. In such a scenario, skip to the Upgrade section.

However, if your environment consists multiple servers in terms of 2xCAS or 2xMBX or 2x(CAS+MBX) or any of a combined scenario, it is highly advised that you perform upgrading of the servers one by one. This will ensure that your environment will stay online using the rest of the high available servers taking the downtime away.

There’s a catch in here. Let’s say we have a 2 Mailbox Server scenario: Server A and B. You decide to mount all databases into ServerA and upgrade Server B. Although this seems an okay way to do, it is not the case. Reason is that the Active Manager and other Exchange backend workers doesn’t know that you are doing a planned maintenance. Thus, the workers will think that the server is in a failed state. Can we fix this? Absolutely. You have to take the services on Server B or the server that you are about to do the upgrade into ‘Maintenance Mode’.

Below steps will guide you to do so;

  1. Run Exchange PowerShell as Administrator.
  2. Run below command to drain connections on the Hub Transport service
    Set-ServerComponentState <SERVER> -Component HubTransport -State Draining -Requester Maintenance

  3. Run below command to disable cluster services
    Suspend-ClusterNode <SERVER>

  4. Run below command to prevent databases from being mounted on the server. It will also immediately move any mounted databases on the server to other servers if copies exist and are healthy. 
    Set-MailboxServer <server> -DatabaseCopyActivationDisabledAndMoveNow $true

  5. Verify that no mailbox databases are mounted on the server.Get-MailboxDatabaseCopyStatus
  6. Run below command to block databases being automatically activated on the specified Mailbox server.
    Set-MailboxServer <server> -DatabaseCopyAutoActivationPolicy Blocked

  7. Run below command to put the state of all components together to Inactive.
    Set-ServerComponentState <server> -Component ServerWideOffline -State Inactive -Requester Maintenance

  8. Close PowerShell.

Upgrade Schema

Now that the server has been put to maintenance mode and you are ready to proceed with the Upgrade process. First thing we need to do is, update the schema. This is critical because without the required schema updates, the setup will not proceed.

  1. Run command prompt as Administrator.
  2. Run below command
    Setup.exe /PrepareSchema /IAcceptExchangeServerLicenseTerms

  3. Run below command
    Setup.exe /PrepareAD /IAcceptExchangeServerLicenseTerms

  4. Run below command
    Setup.exe /PrepareAllDomains/IAcceptExchangeServerLicenseTerms
OR

Setup.exe /PrepareDomain/IAcceptExchangeServerLicenseTerms

  5. Navigate to the Exchange installation binaries folder.
  6. Run setup.exe.
  7. On the Check for Updates window, select your choice and click OK.
  8. On the Upgrade window, click Next.
  9. On the License Agreement page select “I accept ….” and click Next.
  10. On the Readiness Check page, ensure no prerequisites are pending. Click Next to proceed.
  11. Now the installation will proceed.
  12. Once the installation is finished, click Finish.
  13. Restart the server.

 

Now that the server has been upgraded successfully we can resume Exchange services back to normal mode. Follow the below steps.

  1. Run Exchange PowerShell as Administrator.
  2. Run below command to enable connections on the Hub Transport service.
    Set-ServerComponentState <SERVER> -Component HubTransport -State Active -Requester Maintenance

  3. Run below command to enable cluster services
    Resume-ClusterNode <SERVER>

  4. Run below command to enable databases from being mounted on the server.
    Set-MailboxServer <server> -DatabaseCopyActivationDisabledAndMoveNow $false

  5. Run below command to enable databases being automatically activated on the specified Mailbox server.
    Set-MailboxServer <server> -DatabaseCopyAutoActivationPolicy Unrestricted

  6. Run below command to put the state of all components together to Activate.
    Set-ServerComponentState <server> -Component ServerWideOffline -State Active -Requester Maintenance

  7. Close PowerShell.

Now that your server is back in the game. It’s time to upgrade the second server. Perform the steps above from the beginning to do so.

Happy Upgrading!! J

by judeperera

Can I Add Remove Exchange Server 2013 Roles ? Watch out!

With the all new architecture, Exchange Server 2013 RTM now only has 2 Primary Server Roles; Client Access and Mailbox. This however took a slight change of path with the Messaging team finally deciding to add Edge Transport role to the stack as well. Still, comparing to the previous Exchange Server versions, this is a totally different architecture in terms of almost everything.

One major thing that someone should very well focus on is the design. You should be pretty much sure on what roles that you are going to assign for your servers with the options of;

  • Install Standalone Server Roles*
  • Combine Server Roles

*Edge role cannot be collocated/combined with any other server role.

Now even with the above option, there is a tricky point for you when it comes to installing and uninstallation of the roles.

  1. You CAN install one exchange server role (MBX/CAS), and later add the other  role(CAS/MBX) to the existing server.

add

BUT

  1. You CANNOT remove a server role  in a multi-role server with both CAS and MBX installed.

remove

This scenario should be taken quite seriously since there should be no room for any changes once you install both roles. So let’s say what if you came across in a situation like this? Well, the only option is to remove/uninstall the server completely and re-deploy. Which! is going to be a pain for sure!

 

The above is valid for the following Exchange Server 2013 versions as mentioned here;

  • Exchange Server 2013 RTM
  • Exchange Server 2013 CU 1
  • Exchange Server 2013 CU 2

If it’s still the same for CU 3 and SP1? Well, I shall post ASAP and if you were able to check it out, let me know.

Cheers!!!

 

by judeperera

Step by Step Guide for Installing Exchange Server 2013 Preview

Update(2014-11-07): Covers the RTM installation.

The following section describes a step-by-step guide for the installation of Microsoft® Exchange Server 2013 Preview. The installation considers a single server deployment of Exchange Server 2013 with the Mailbox and Client Access Server roles collocated. Additional details of the topology and architecture of the lab environment which was used in the installation is described here;

Continue reading →

by judeperera

Lync Server 2013 Preview Download Available Now

It indeed has been a wonderful day for me with Microsoft. And that’s true for all IT Pro’s as well who are eager to test drive the next version of their skilled technology. With Office 2013 going as a ppublic beta, Microsoft has made Exchange Server 2013, Lync Server 2013 and SharePoint 2013 Preview versions as well for us to play before everything gets RTM’ed.

And for all Lync enthusiasts, they can get the download of the latest public preview right from the following image (click to download) which requires you to sign-in using a Live-ID.

Additionally, the following material are available online for references as well;

 

Want more?

Check out the official Lync 2013 Preview page over at http://lync.microsoft.com/en-us/Pages/Lync-2013-Preview.aspx

Happy Testing everyone 🙂