April 20, 2021 by judeperera

Exchange Server 2019 and Office Web App Server Error: “Sorry, there was a problem and we can’t open this document”

If you are using Exchange Server 2019, running on Windows Server 2019 and Office Online Server (Office Web Apps) running on Windows Server 2016 or older operating systems, you will get an error as shown below:

“Sorry, there was a problem and we can’t open this document. If this happens again, try opening the document in Microsoft Word.”

Step 01: Enable OOS ULS Logging

To troubleshoot the above, we need to enable logging on the Office Online Server (OOS). Please follow the steps to enable logging.

Open up PowerShell and run the below command to enable logging.

Set-OfficeWebAppsFarm -LogVerbosity “High”
Restart the Office Online service by running the below command.

Restart-Service WACSM -force

Step 02: Check Logs

Upon going through the logs, map your Session ID with the log lines. In my case, the error noted a .NET exception where the connection was forcibly closed by the remote host, which is the Exchange Server.

ServiceHostLoader.GetCheckedBaseDocument: Caught a FileUnknownException: Microsoft.Office.Web.Common.EnvironmentAdapters.FileUnknownException: WOPI Check File —> Microsoft.Office.Web.Common.EnvironmentAdapters.UnexpectedErrorException: HttpRequest failed. —> Microsoft.Office.Web.Common.HttpRequestAsyncException: No Response in WebException —> System.Net.WebException: The underlying connection was closed: An unexpected error occurred on a send. —> System.IO.IOException: Unable to read data from the transport connection: An existing connection was forcibly closed by the remote host. —> System.Net.Sockets.SocketException: An existing connection was forcibly closed by the remote host

Step 03: Enable TLS and .NET Strong Encryption

Exchange Server 2019 only supports TLS v1.2 along with enhanced ciphers and hashing algorithms. Since the older operating systems may not be enabled with these settings, we need to manually enable them through registry.

Verify that same TLS versions are enabled on both Office Online Server and Exchange Servers
Backup the following registry paths
1. HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\.NETFramework\v4.0.30319
2. HKEY_LOCAL_MACHINE\SOFTWARE\Wow6432Node\Microsoft\.NETFramework\v4.0.30319
Change/Add new DWORD using below

[HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\.NETFramework\v4.0.30319]

“SchUseStrongCrypto”=dword:00000001

[HKEY_LOCAL_MACHINE\SOFTWARE\Wow6432Node\Microsoft\.NETFramework\v4.0.30319]

“SchUseStrongCrypto”=dword:00000001
Restart the Office Online service by running the below command.

Restart-Service WACSM -force

That’s it. Now you should be able to see the documents and spreadsheets as expected.

Step 04: Disable OOS ULS Logging

To troubleshoot the above, we need to enable logging on the Office Online Server (OOS). Please follow the steps to enable logging.

Open up PowerShell and run the below command to enable logging.

Set-OfficeWebAppsFarm -LogVerbosity “”
Restart the Office Online service by running the below command.

Restart-Service WACSM -force

March 10, 2021March 12, 2021 by judeperera

Microsoft Releases Critical Exchange Server Security Updates for older CUs

Note: this post may get updated; please keep checking back. Last update: 3/12/2021

In the wake of the recent vulnerability, Microsoft immediately started releasing the updates for the latest supported versions of Microsoft Exchange Server versions 2010,2013,2016 and 2019. However these updates were targeting only the latest (N) and immediate previous (N-1) only. With the growing concerns and the criticality, Microsoft has decided to take a step further from its traditional approach by releasing additional updates for older CUs.

Therefore if you are on below Exchange Server CUs hurry up and start patching your servers ASAP!

Exchange Server 2019

Exchange Server	Standalone Security Update
Exchange Server 2019 CU 8	Download
Exchange Server 2019 CU 7	Download
Exchange Server 2019 CU 6	Download
Exchange Server 2019 CU 5	Download
Exchange Server 2019 CU 4	Download
Exchange Server 2019 CU 3	Download
Exchange Server 2019 CU 2	To be Updated
Exchange Server 2019 CU 1	To be Updated
Exchange Server 2019 RTM	Download

Exchange Server 2016

Exchange Server	Standalone Security Update
Exchange Server 2016 CU 19	Download
Exchange Server 2016 CU 18	Download
Exchange Server 2016 CU 17	Download
Exchange Server 2016 CU 16	Download
Exchange Server 2016 CU 15	Download
Exchange Server 2016 CU 14	Download
Exchange Server 2016 CU 13	Download
Exchange Server 2016 CU 12	Download
Exchange Server 2016 CU 11	Download
Exchange Server 2016 CU 10	Download
Exchange Server 2016 CU 9	Download
Exchange Server 2016 CU 8	Download

Exchange Server 2013

Exchange Server	Standalone Security Update
Exchange Server 2013 CU 23	Download
Exchange Server 2013 CU 22	Download
Exchange Server 2013 CU 21	Download

Step by step update guide – Exploitation of Exchange Server Vulnerability – Notes from the Field P1 | Jude’s Blog (wordpress.com)

Source: March 2021 Exchange Server Security Updates for older Cumulative Updates of Exchange Server – Microsoft Tech Community

https://www.microsoft.com/download/details.aspx?familyid=1a07c860-4149-4a9e-b9cc-6a656a7e8916

March 10, 2021March 10, 2021 by judeperera

Microsoft Security Update Release for March 2021

It’s patch Tuesday!!! Microsoft releases security updates on the second Tuesday of each month to address newly reported security vulnerabilities in Microsoft products. In this release Microsoft has focused on addressing some major vulnerabilities. For the month of March, the updates have been released on the 9^th March, 2021.

Microsoft has fixed 82 vulnerabilities, with 10 classified as Critical and 72 as Important. These numbers do not include the 7 Microsoft Exchange and 33 Chromium Edge vulnerabilities released earlier this month. – BleepingComputer

Happy Patching!!! 🙂

March 7, 2021 by judeperera

Exploitation of Exchange Server Vulnerability – Notes from the Field P1

Note: this post may get updated; please keep checking back. Last update: 3/7/2021

Microsoft, on the very same day of its global event “Ignite 2021” made its headlines globally over a zero-day out of band patch release. The security update was a fix to multiple critical Exchange Server vulnerabilities. At the time of initial information disclosure it was not aware how long the vulnerabilities were known in the wild. However, the MSTIC or the Microsoft Security and Threat Intelligence Center reported the campaign was largely carried out by a state sponsored group “Hafnium”.

Microsoft did make the announcement very clear to it’s partners and customers. Simply, it’s about patching all your on-premise Exchange servers as soon as possible. The vulnerabilities were identified as CVE-2021-26855, CVE-2021-26857, CVE-2021-26858, and CVE-2021-27065. It was also disclosed that Exchange Server versions 2010, 2013, 2016 and 2019 were affected.

However, there was a caveat, behind the updates. You can only patch your servers if you have “a supported version of Exchange Server” only. So what does it mean? Let’s talk about it now. Microsoft in its product lifecycle clearly states specially in terms of Exchange Server that the current version (n) and the immediate previous version (n-1) will be the only supported version unless told. For customers who keep a habit of updating their servers regularly, this is not much of a problem. But if you are to deal in an environment where you are not in a supported version, and more to that haven’t updated since installing your Exchange in the very beginning, it’s going to be some tough nights for you and your IT team.

The vulnerabilities targets your internet facing Client Access Servers. However, Microsoft highly recommends that you install updates on all of your Exchange Servers immediately even if they are;

Internet facing
Non-internet facing
Hybrid servers
Behind a proxy or WAF solution

PATCH YOUR SERVERS!!!

During my walk through on how to install these updates, I will be taking in two scenarios. For your ease, here’s a flowchart on the highlights.

Exchange Updated with the latest public build
Exchange Not-updated with the latest supported build

If you are not quite sure to which category you belong to, use the below table to identify if you have the latest build.

Exchange Server Version	Exchange PowerShell Command	Supported build number	Download the Supported Build
Exchange Server 2010	Get-Command ExSetup \| ForEach {$_.FileVersionInfo}	14.3.123.4 and above	Exchange Server 2010 SP3
Exchange Server 2013	Get-ExchangeServer \| Format-List Name,Edition,AdminDisplayVersion	15.0.1497.2 and above	Exchange Server 2013 CU23
Exchange Server 2016	Get-ExchangeServer \| Format-List Name,Edition,AdminDisplayVersion	15.1.2106.2 and above	Exchange Server 2016 CU19
Exchange Server 2019	Get-ExchangeServer \| Format-List Name,Edition,AdminDisplayVersion	15.2.792.3 and above	Exchange Server 2019 CU8

Exchange Updated with the latest public build

You are in a supported build number and that means things will be easy for you. Let’s go through the steps one by one. Thankfully Microsoft only released one patch that addresses all the four vulnerabilities. This means less installation and reboots.

When the security experts are emphasizing on this its for your own good that you might want to call your teams and get this done right away with some severe convincing to your management if they are reluctant. Afterall, explaining why you are taking a few hours of downtime is far more better than explaining why your company data is out on the dark web.

Pre-requisites. Make sure you do these.
1. Backup your Exchange Servers.
2. Backup your customized themes, logos etc. (Customize the Outlook on the web sign-in, language selection, and error pages in Exchange Server | Microsoft Docs)
Download the correct patch file based on your Exchange Server version

Exchange Server Version	Exchange PowerShell Command
Exchange Server 2010 (SP 3 or above)	KB5000978
Exchange Server 2013 CU 23	KB5000871
Exchange Server 2016 CU 18	KB5000871
Exchange Server 2016 CU 19	KB5000871
Exchange Server 2019 CU 7	KB5000871
Exchange Server 2019 CU 8	KB5000871

Open up a command prompt as Administrator.
Navigate to the path where you downloaded the patches.
Type in the name of the .msp file, and then press Enter.
If you require any pre-requisites, you will be notified.
Once your installation is completed, you may be asked to restart the server. In case you are not prompted to, still it’s better to restart the server.
Repeat the above steps for all your remaining Exchange Servers.

Exchange Not-updated with the latest public build

There’s going to be a few additional steps that you will have to carry out if your Exchange Server is not in the supported list as highlighted above. So the very first thing you need to do is to bring the server to the latest supported build.

Update servers to the latest build

Pre-requisites. Make sure you do these.
1. Backup your Exchange Servers.
2. Backup your customized themes, logos etc. (Customize the Outlook on the web sign-in, language selection, and error pages in Exchange Server | Microsoft Docs)
Exchange latest CUs may require the latest .NET version. So you need to verify if your current .NET version (how to find my .NET version?) is different from the supported .NET versions as highlighted below, you need to download the latest supported .NET Framework and install it before proceeding.

More details – Exchange Server supportability matrix | Microsoft Docs
Once you’re done, my personal advice is to compile the .NET binaries before the CU update. The reason why I’m saying this is I’ve seen multiple times that not doing so makes the installation time go through several hours and doing so drastically reduced the time of the installation. To do that;
1. Open up command prompt and run as administrator.
2. Navigate to “C:\Windows\Microsoft.NET\Framework\v4. 0.30319\”
3. Run “ngen.exe update” (without quotes)
Note: this will run for a while and will give you various outputs on the command prompt, don’t worry you can ignore them all.
Once the compile is run, exit the command prompt.
Download the correct update binaries based on your Exchange Server version

Exchange Server Version	Exchange PowerShell Command
Exchange Server 2010 (SP 3 or above)	KB5000978
Exchange Server 2013 CU 23	KB5000871
Exchange Server 2016 CU 18	KB5000871
Exchange Server 2016 CU 19	KB5000871
Exchange Server 2019 CU 7	KB5000871
Exchange Server 2019 CU 8	KB5000871

Run the setup and extract to a location on your computer.
Open up a command prompt as Administrator.
Navigate to the path where you downloaded and extracted the update files.
Prepare your Active Directory
1. Run “E:\Setup.exe /IAcceptExchangeServerLicenseTerms /PrepareSchema” to extend the Active Directory schema.
2. Run “E:\Setup.exe /IAcceptExchangeServerLicenseTerms /PrepareAD” to extend the AD.
Run “SETUP.EXE” to start Exchange CU update.
Go through the Exchange Update Wizard.
Once your installation is completed, restart the server.
Repeat the above steps for all your remaining Exchange Servers.
Once you have completed updating all your servers to the latest supported Exchange Server build, continue with the steps mentioned in the “Exchange Updated with the latest public build”

Keep us posted on any errors you come across. Happy updating fellas!!

June 23, 2020 by judeperera

[New Release] Recover deleted items using M365 Exchange Admin Center

Users accidentally deleting emails can sometimes be a painstaking task for an administrator. Specially if you are in charge of 1000+ users and majority of them are not tech savvy. For the past few years Office 365, or Microsoft 365 as now we call it gave us two main options. First would be to use the Outlook client and the second would be to use PowerShell.

Finally, Microsoft has released the ability to “Recover deleted items” using the New M365 Exchange Admin Center.

Prerequisites

The Exchange administrator account that will be used to recover the mail items needs to be assigned with the “Mailbox Import Export” permission. To do this, you can either create a new Role or modify an existing role and add the “Mailbox Import Export” to the permission list.
Once permissions are granted, wait for a few minutes, sign-off and sign-in again for the permissions to be active for your session
This will adhere to your deleted items retention policies placed within the organization or user level.

For the below demo, we’ll look at the below users Deleted Items folder.

Now that we know what items will be recovered, let’s head to the Exchange Admin Center and see how the recovery is done.

Login to the all-new Exchange Admin Center : https://admin.exchange.microsoft.com
Navigate to Recipients > Mailboxes
Select the user that you wish to recover the items from, click on the three dots (more options) and select Recover deleted items.
Once you are in the Recover deleted items page, adjust the filters as you wish and click Apply filter. As you can see, all mail items are visible in the search results that’s also visible in the users Deleted Items folder from his personal login.
Now to recover, you can either select a single entry, multiple entries or all items and click on the Recover deleted items button.
The email item will now be restored to the Original folder as displayed in the above result summary.

And you are done!

Microsoft has really worked this feature out in terms of user-friendliness with the new EAC. This is just one feature and in the future posts, I will demo the ret of the new additions. Give this a try and share your thoughts.

May 18, 2020 by judeperera

Microsoft Teams Attendance Report is here

The long await is finally over. Yes, Microsoft has rolled out the much-awaited participation report for a meeting. This feature is long due as getting an attendance report is indeed a must in a meeting specially for Educators. The feature has now come up as a workaround to general availability.

“Thank you for the feedback and waiting for this feature with patience. We have enabled the shortterm fix to allow download of a meeting attendance list during a meeting (from the roster view). This is released for general availability.” – Alex (Teams Engineering, Microsoft Teams)

Now let’s see the look and feel;

For this to work, you need to be the meeting organizer. If you are, while you are on the meeting, navigate to the peoples pane and check the top right corner section and you will see the Download icon.

However, there is a catch as of now. The participation list download will only be available while the meeting is going on for the meeting organizer only. So if you are the organizer you might want to download the attendance list right before you exit the call.

That’s it. It’s that simple.

For Administrators

However, if you are an administrator and would like to see the Enabled or Disabled status of the feature to your tenant. You can use the below methods.

Step 01: Open your PowerShell and type

Import-Module SkypeOnlineConnector

Step 02: Run the below command and enter your tenant administrator credentials;

$userCredential = Get-Credentials

Step 03: Create connection to CS Online modules using the below commands;

$sfbSession = New-CsOnlineSession -Credential $userCredential

Step 04: Import CS PowerShell session

Import-PSSession $sfbSession

Step 05: Now that we are connected to the CS online for your tenant, let’s use the below command to view the Global policy settings;

Get-CsTeamsMeetingPolicy -identity Global

Keep a note on the setting highlighted above AllowEngagementReport. In our case this setting is already Enabled. Microsoft is rolling out this feature Enabled by default so you shouldn’t need to worry about it. However, if you are unable to see it yet, you might want to have a look at this setting.

Step 06: In case your setting is shown as Disabled or you want to change it, you can use the below command;

Set-CsTeamsMeetingPolicy -Identity Global -AllowEngagementReport Enabled

That’s it! Your new settings will now be applied. Additionally, you might want to try below options if you still cant see;

Sign-out and sign back in.
Update your Teams app
Wait: yes, give it few days as the global roll-out has already started and it will come to you soon.

October 28, 2019 by judeperera

Exchange ECP Empty Results Issue in Internet Explorer

When it comes to browser compatibility issues, it’s always about non-Microsoft browsers having issues with Exchange Servers. But in this particular case, the issue was vise-versa.

This was an Exchange 2013 environment, and the administrator reported that while using the ECP in Internet Explorer he was unable to view any data within sub windows and the links within were not clickable. However, when using Google Chrome or FireFox, this was not a problem.

The above image is what you get when you open a Recipient mailbox in ECP. As you can see, the values are all empty and I cannot click any of the action items on left side.

Now let’s see what type of an approach you should take in a scenario like this.

Check with multiple Internet Explorer versions/users. Plus clear your cache and cookies as well.
Check with compatibility mode on IE.
Check if the issue is for both internal and external users. Why I’m saying this is that in most of the cases your Exchange Servers may be placed behind many layers such as proxy, load balancers etc. So, we need to narrow this down to internal or external.
Check if the issue is for a single server only. Go to your exchange server, open Internet Explorer and check your ECP using localhost FQDN. By this, we can see if the issue is only for one Exchange Server or multiple servers.
Now that you have figured out where and which servers are affected, it’s time to refresh your IIS cache. We do this because that’s the first point where all web connections hit to. And I’ve seen that in most client connectivity issues, doing a simple refresh or may be a reset on IIS fixes the issue. Resetting IIS would be the last resort as it would drop all connections and users will not be able to connect until the service starts which can take some time.

Below are the steps that worked for me;

Open IIS Manager and navigate to Application
Pools.
Under the list of pools, you can see the MSExchangeECPAppPool. This is responsible for ECP connections and we can easily refresh the cache.
Right click the MSExchangeECPAppPool and click Recycle.
Now IIS will refresh your specific app pool.
Login back to the ECP and check if the issue is sorted.
In my case, it was working again.

October 16, 2019 by judeperera

Exchange 2016 CU Update error at “Mailbox role: Transport service”

Well, this was a strange case that came across while installing the Exchange Server 2016 CU14 update. The setup goes all the way and throws an error at “Mailbox role: Transport service” component installation and exits.

Issue:

Error:

The following error was generated when "$error.Clear();

if ($RoleProductPlatform -eq "amd64")

{

try

{

# Need to configure the ETL traces before the fast service is installed. This will ensure that when the service comes up

# it will have the necessary trace session setting available to read from the registry

$fastPerfEtlTraceFolderPath = Join-Path -Path $RoleBinPath -ChildPath "\Search\Ceres\Diagnostics\ETLTraces"

$fastDiagnosticTracingRegKeyPath = 'HKLM:\SOFTWARE\Microsoft\Office Server\16.0\Search\Diagnostics\Tracing'

if(-not(Test-Path -Path $fastPerfEtlTraceFolderPath))

{

$null = New-Item $fastPerfEtlTraceFolderPath -Type 'Directory' -Force

}

if (-not(Test-Path -Path $fastDiagnosticTracingRegKeyPath))

{

$null = New-Item -Path $fastDiagnosticTracingRegKeyPath -Force

}

$null = New-ItemProperty -Path $fastDiagnosticTracingRegKeyPath -Name 'TracingPath' -PropertyType 'string' -Value $fastPerfEtlTraceFolderPath -Force

$null = New-ItemProperty -Path $fastDiagnosticTracingRegKeyPath -Name 'TracingFileName' -PropertyType 'string' -Value 'DocumentProcessingTrace' -Force

$null = New-ItemProperty -Path $fastDiagnosticTracingRegKeyPath -Name 'DocumentParserSuccessLogMessage' -PropertyType 'Dword' -Value 1 -Force

$null = New-ItemProperty -Path $fastDiagnosticTracingRegKeyPath -Name 'DocumentParserLoggingNoInitialisation' -PropertyType 'Dword' -Value 1 -Force

# Max trace folder size 50 * 100 = 5GB

$null = New-ItemProperty -Path $fastDiagnosticTracingRegKeyPath -Name 'MaxTraceFileSize' -PropertyType 'Dword' -Value 50 -Force

$null = New-ItemProperty -Path $fastDiagnosticTracingRegKeyPath -Name 'MaxTraceFileCount' -PropertyType 'Dword' -Value 100 -Force

$null = New-ItemProperty -Path $fastDiagnosticTracingRegKeyPath -Name 'UseGeneralSwitch' -PropertyType 'Dword' -Value 1 -Force

$null = New-ItemProperty -Path $fastDiagnosticTracingRegKeyPath -Name 'GeneralSwitch' -PropertyType 'Dword' -Value 0 -Force

}

catch

{

# ETl tracing is not critical. Info only log

Write-ExchangeSetupLog -Info ("An exception ocurred while trying to Configure the FAST ETL traces. Exception: " + $_.Exception.Message);

}

try

{

$fastFusionRegKeyPath = 'HKLM:\SOFTWARE\Microsoft\Office Server\16.0\Search\FlightControl'

if (Test-Path -Path $fastFusionRegKeyPath)

{

Remove-ItemProperty -Path $fastFusionRegKeyPath -Name 'fusion_new_enabled' -Force -ErrorAction SilentlyContinue

Remove-ItemProperty -Path $fastFusionRegKeyPath -Name 'fusion_old_enabled' -Force -ErrorAction SilentlyContinue

Remove-ItemProperty -Path $fastFusionRegKeyPath -Name 'fusion_compare_outputs' -Force -ErrorAction SilentlyContinue

}

}

catch

{

# Removing new fusion keys is not critical. Info only log

Write-ExchangeSetupLog -Info ("An exception ocurred while trying to remove the fast new fusion reg keys. Exception: " + $_.Exception.Message);

}

$fastInstallConfigPath = Join-Path -Path $RoleBinPath -ChildPath "Search\Ceres\Installer";

$command = Join-Path -Path $fastInstallConfigPath -ChildPath "InstallConfig.ps1";

$dataFolderPath = Join-Path -Path $RoleBinPath -ChildPath "Search\Ceres\HostController\Data";

# Remove previous SearchFoundation configuration

&$command -action u -silent;

try

{

if ([System.IO.Directory]::Exists($dataFolderPath))

{

[System.IO.Directory]::Delete($dataFolderPath, $true);

}

}

catch

{

$deleteErrorMsg = "Failure cleaning up SearchFoundation Data folder. - " + $dataFolderPath + " - " + $_.Exception.Message;

Write-ExchangeSetupLog -Error $deleteErrorMsg;

}

# Re-add the SearchFoundation configuration

try

{

# the BasePort value MUST be kept in sync with dev\Search\src\OperatorSchema\SearchConfig.cs

&$command -action i -baseport 3800 -dataFolder $dataFolderPath -silent;

}

catch

{

$errorMsg = "Failure configuring SearchFoundation through installconfig.ps1 - " + $_.Exception.Message;

Write-ExchangeSetupLog -Error $errorMsg;

# Clean up the failed configuration attempt.

&$command -action u -silent;

try

{

if ([System.IO.Directory]::Exists($dataFolderPath))

{

[System.IO.Directory]::Delete($dataFolderPath, $true);

}

}

catch

{

$deleteErrorMsg = "Failure cleaning up SearchFoundation Data folder. - " + $dataFolderPath + " - " + $_.Exception.Message;

Write-ExchangeSetupLog -Error $deleteErrorMsg;

}

}

# Set the PowerShell Snap-in's public key tokens

try

{

$PowerShellSnapinsPath = "HKLM:\SOFTWARE\Microsoft\PowerShell\1\PowerShellSnapIns\";

$FastSnapinNames = @("EnginePSSnapin", "HostControllerPSSnapIn", "InteractionEnginePSSnapIn", "JunoPSSnapin", "SearchCorePSSnapIn");

$officePublicKey = "71E9BCE111E9429C";

$exchangePublicKey = "31bf3856ad364e35";

foreach ($fastSnapinName in $FastSnapinNames)

{

$fastSnapinPath = $PowerShellSnapinsPath + $fastSnapinName;

$assemblyNameProperty = Get-ItemProperty -Path $fastSnapinPath -Name "AssemblyName" -ErrorAction SilentlyContinue;

if ($assemblyNameProperty -ne $null -and (-not [string]::IsNullOrEmpty($assemblyNameProperty.AssemblyName)))

{

$newAssemblyName = $assemblyNameProperty.AssemblyName -ireplace ($officePublicKey, $exchangePublicKey);

Set-ItemProperty -Path $fastSnapinPath -Name "AssemblyName" -Value $newAssemblyName;

}

}

}

catch

{

# Info only log

Write-ExchangeSetupLog -Info ("An exception ocurred while configuring Search Foundation PowerShell Snapin. Exception: " + $_.Exception.Message);

}

}

" was run: "System.Exception: Failure configuring SearchFoundation through installconfig.ps1 - Error occurred while configuring Search Foundation for Exchange.System.TimeoutException: This request operation sent to net.tcp://sandesha-b.cbsl.lk:3803/Management/AdminService did not receive a reply within the configured timeout (00:01:00). The time allotted to this operation may have been a portion of a longer timeout. This may be because the service is still processing the operation or because the service was unable to send a reply message. Please consider increasing the operation timeout (by casting the channel/proxy to IContextChannel and setting the OperationTimeout property) and ensure that the service is able to connect to the client.

Server stack trace:

at System.ServiceModel.Dispatcher.DuplexChannelBinder.Request(Message message, TimeSpan timeout)

at System.ServiceModel.Channels.ServiceChannel.Call(String action, Boolean oneway, ProxyOperationRuntime operation, Object[] ins, Object[] outs, TimeSpan timeout)

at System.ServiceModel.Channels.ServiceChannelProxy.InvokeService(IMethodCallMessage methodCall, ProxyOperationRuntime operation)

at System.ServiceModel.Channels.ServiceChannelProxy.Invoke(IMessage message)

Exception rethrown at [0]:

at System.Runtime.Remoting.Proxies.RealProxy.HandleReturnMessage(IMessage reqMsg, IMessage retMsg)

at System.Runtime.Remoting.Proxies.RealProxy.PrivateInvoke(MessageData& msgData, Int32 type)

at Microsoft.Ceres.CoreServices.Admin.IAdminServiceManagementAgent.UpdateConfiguration()

at Microsoft.Ceres.Exchange.PostSetup.NodeManager.AddNodeAndUpdateConfiguration(String node)

at Microsoft.Ceres.Exchange.PostSetup.NodeManager.DeployContentEngineNode()

at Microsoft.Ceres.Exchange.PostSetup.DeploymentManager.Install(String installDirectory, String dataDirectoryPath, Int32 basePort, String logFile, Boolean singleNode, String systemName, Boolean attachedMode)

at CallSite.Target(Closure , CallSite , RuntimeType , Object , Object , Object , Object , Object , Object , Boolean )

at Microsoft.Exchange.Configuration.Tasks.Task.ThrowError(Exception exception, ErrorCategory errorCategory, Object target, String helpUrl)

at Microsoft.Exchange.Management.Deployment.WriteExchangeSetupLog.InternalProcessRecord()

at Microsoft.Exchange.Configuration.Tasks.Task.<ProcessRecord>b__91_1()

at Microsoft.Exchange.Configuration.Tasks.Task.InvokeRetryableFunc(String funcName, Action func, Boolean terminatePipelineIfFailed)".

Resolution:

We actually made a few changes from the server end and once we tried again, the setup ran without any issues. Therefore if you encounter such an error above, you might want to try this out and see if any of these will work;

Run the setup again and check
Check with Anti-virus software is disabled
Check with Firewall disabled
Check with setting the PowerShell Execution Policy of the server to ‘Unrestricted’ (Set-ExecutionPolicy Unrestricted)

Once you have done the above, give the installer another try and hopefully this will fix. In my case, I was able to finish the installation after doing the last three mentioned.

Happy Installations!!

October 11, 2019 by judeperera

Fixing “451 4.7.0 Temporary server error. Please try again later. PRX4”

Issue

In this scenario we are going to talk about an issue related to Exchange server not being able to accept any inbound SMTP connections. The client had one Edge server and two Exchange 2016 Mailbox servers. Out of nowhere, they have noticed that they are unable to send emails within the Exchange organization as well as all inbound emails from internet seemed to have stopped. In such a scenario, what we should do first is to identify the role that the issue might be at. In our case this is to with the SMTP mail flow thus the Hub Transport service should be involved.

Also, looking at the mail queues in the gateway and exchange servers, the Last Known Error message is shown as below;

451 4.7.0 Temporary server error. Please try again later. PRX4

Step 01:
Our first step should be to make sure that all Exchange Services are running on the servers. You can do this by going to the “Services” or opening up Exchange Powershell and running the below command to ensure that all required services are in “Running” state.

Get-ServerComponentState -Identity <ServerIdParameter>

Step 02:
Now that we know our services are working (at least showing that it’s working) let’s do a telnet and verify. You can do this by sending an email using ‘Telnet‘ from your other hops. In my case;

Edge server to Exchange server 01 – Gives error “451 4.7.0 Temporary server error. Please try again later. PRX4”
Exchange server 01 to Exchange server 01 – Gives error “451 4.7.0 Temporary server error. Please try again later. PRX4”
Exchange server 01 to Exchange server 01 – Gives error “451 4.7.0 Temporary server error. Please try again later. PRX4”
Exchange server 02 to Exchange server 02 – Gives error “451 4.7.0 Temporary server error. Please try again later. PRX4”
Exchange server 02 to Exchange server 01 – Gives error “451 4.7.0 Temporary server error. Please try again later. PRX4”
Exchange server 02 to Edge server 01 – Message relayed successfully
Exchange server 02 to Edge server 02 – Message relayed successfully

What does this say? Issue is on my internal Exchange Servers, ditto!

Step 03: Now that we have established where the issue occurs on which service, the next option was to check the AD connectivity. You can do this by checking the event log for “MSExchange
ADAccess” and confirm that Exchange and AD connection is working as expected.

Step 04: Check if the time is correct on all your Exchange servers along with the time of your AD.

Step 05: Verify that the entries in your DNS is correct and working by doing a NSLOOKUP from all Exchange servers.

Step 06: Verify that your certificates are valid and SMTP services are assigned properly.

In my scenario, all steps from 1-5 were perfectly fine except for step 6. I logged into my ECP, went to Servers and Certificates and in my list, I noticed that the certificate Status was showing as ‘Invalid‘. In addition, the certificate properties showed that the SMTP service was also assigned to this certificate.

Resolution

We can get an idea on what might the issue be. We have an Invalid certificate with SMTP assigned to it. And our issue is that the Exchange server is rejecting the SMTP connections. This seems to be a valid cause. Now check your other certificates too. In my case the certificate in issue was an older custom created one. But there was a proper certificate that was set to IIS,SMTP,POP,IMAP services. That means we can delete it. However, if the certificate is a built-in certificate or the only certificate that’s assigned to all services, ‘DO NOT DELETE’.

Before deleting, it’s always a good idea to take a backup of the certificate. To do this, follow the steps;

Open MMC
Click File and select Add/Remove Snap In
Under the Available
snap–ins, click on Certificates and Add
Under Certificates snap-in select ‘Computer
account’ and click Finish
Select Local Computer to manage the snap-in
Click OK to add the selected snap-in to console window
Go to the MMC, under Personal Certificates store, right-click on your certificate that should be exported, and select All Tasks and click Export
Proceed with exporting the certificate with the Private key.

Now that we have exported the certificate, we can go back to the ECP and delete the Invalid certificate.

Login to Exchange
Admin
Center (ECP)
Navigate to Servers and click on Certificates
Select the server you want to list the certificates from the drop-down menu
Select the certificate which is marked as Invalid that you want to delete
Click on the delete icon (recycle-bin icon)
If you have multiple servers, do steps 2-5

The next step is to restart the Microsoft Exchange Transport related services so that the service will now refresh its certificate and will only use the Valid certificate(s). Do this on all servers that you removed the certificate.

Give your services to run and the mail queues to automatically restart and you will be able to see that the mail Queue will now be delivered.

September 3, 2019September 18, 2019 by judeperera

[Script] Shared Mailbox Convert and Enabling Sent Items Copy

This is a simple script that will allow an administrator to achieve the below tasks;

Convert a User mailbox to a Shared Mailbox
Enable Messages sent from the shared mailbox to be saved to the Sent Items folder of the shared mailbox

Let’s take the below scenario;

User Mailbox – user@contoso.com
Shared Mailbox – shared@contoso.com (user@contoso.com has SendAs and FullAccess permissions)

In a case where if user(user@contoso.com) tries to send an email with SendAs/SendOnBehalfOf permissions for ‘shared@contoso.com‘, the sent email will only be available in the primary users ‘Sent Items‘ ONLY. In a business requirement where the mail should also be present in the ‘shared@contoso.com‘ shared mailbox, we would require the below permissions to be enabled. Once done, the email with SendAs/SendOnBehalfOf permission will be in both user@contoso.com and shared@contoso.com mailboxes’ Sent Items.

The MessageCopyForSendOnBehalfEnabled parameter specifies whether to copy the sender for messages that are sent from a mailbox by users that have the “send on behalf of” permission. In this script we will be enabling this by setting the value to $true; where when a user sends a message from the mailbox by using the “send on behalf of” permission, a copy of the message is sent to the sender’s mailbox.

The MessageCopyForSentAsEnabled parameter specifies whether to copy the sender for messages that are sent from a mailbox by users that have the “send as” permission. In this script we will be enabling this by setting the value to $true; where when a user sends a message from the mailbox by using the “send as” permission, a copy of the message is sent to the sender’s mailbox.

Want to give a try? you can get the script here..