CSS: How To Set Color Gradient and Animation to Text and Background

// Static color gradient
.colorGradientClass {
    background-color: #ffffff !important;
    background-image: linear-gradient(315deg, #ffffff 0%, #d9d9d9 74%)  !important;
}

// Animated color gradient
.colorTransitionClass {
    background-image: 
        linear-gradient(to right, transparent, white),
        linear-gradient(to right,yellow, white, yellow, white);
    background-size: 100% 100%, 2000% 100%;
    animation: move 5s infinite;
}
@keyframes move {
 from {background-position: center center, left center;}
 to {background-position: center center, right center;}
}

// This will result in blinking text
.blinkingText {
    animation: blinker 5s linear infinite;
    color: red !important;
}
@keyframes blinker {
  50% {
    opacity: 0;
  }
}

How To Move WordPress Site To Kubernetes Cluster

a. Create backups of source files and database

  - Logon to Current Hosting Provider to make backups
  - Files:
    - Assuming cPanel:
      - Login to cPanel
      - Click on 'File Manager'
      - Select public_html or the directory containing WordPress files
      - Select Compress from the top-right menu
      - Select 'Bzip2ed Tar Archive' (better compression than Gzip)
      - Click 'Compress File(s)' and wait for the process to finish
      - Right-click the newly generated public_html.tar.bz2 from cPanel File Manager > select Download
      - Find the file in a default download directory (e.g. /home/$(whoami)/Downloads/public_html.tar.bz2)
  - Database:
    - Assuming cPanel with phpMyAdmin
      - Click 'phpMyAdmin' from the 'DATABASES' control group
      - Click 'Export'
      - Set Export method = Quick, Format = Custom
      - Click Go
      - Find the *.sql file being downloaded into a default download directory (e.g. /home/$(whoami)/Downloads/localhost.sql)

b. Install Bitnami WordPress in a Kubernetes Cluster

# Add helm chart if not already available
helm repo add bitnami https://charts.bitnami.com/bitnami

# Install WordPress with Dynamic NFS Provisioning
# Documentation: https://hub.kubeapps.com/charts/bitnami/wordpress/10.0.1
# Set variables
appName=kimconnectblog
domainName=blog.kimconnect.com
wordpressusername=kimconnect
wordpressPassword=SOMEPASSWORDHERE
rootPassword=SOMEPASSWORDHERE2
storageClass=nfs-client
# Install
helm install $appName bitnami/wordpress \
  --set persistence.accessMode=ReadWriteMany,persistence.storageClass=nfs-client \
  --set mariadb.primary.persistence.storageClass=nfs-client \
  --set wordpressUsername=$wordpressusername,wordpressPassword=$wordpressPassword \
  --set mariadb.auth.rootPassword=$rootPassword \
  --set mariadb.auth.password=$rootPassword \
  --set ingress.enabled=true,ingress.hostname=$domainName
# Patch the deployed ingress with an existing SSL cert
# Assuming the $appName-cert has already been generated
appName=kimconnectblog
domainName=blog.kimconnect.com
certName=$appName-cert
serviceName=$appName-wordpress
servicePort=80
cat <<EOF > $appName-patch.yaml
spec:
  tls:
  - hosts:
    - $domainName
    secretName: $certName
  rules:
  - host: $domainName
    http:
      paths:
      - path: /
        pathType: Prefix
        backend:
          service:
            name: $serviceName
            port:
              number: $servicePort         
EOF
kubectl patch ingress/$appName-wordpress -p "$(cat $appName-patch.yaml)"

c. Import files and database onto new hosting server

  - Database:
    - Access DB server and import sql dump
      podName=kimconnectblog-mariadb-0
      kubectl exec --stdin --tty $podName -- /bin/bash
      rootPassword=SOMEPASSWORD
      echo "show databases;" | mysql -u root -p$rootPassword
      MariaDB [(none)]> show databases;exit;
        +--------------------+
        | Database           |
        +--------------------+
        | bitnami_wordpress  |
        | information_schema |
        | mysql              |
        | performance_schema |
        | test               |
        +--------------------+
        5 rows in set (0.009 sec)
      oldDb=kimconne_blog
      sqlDump=/bitnami/mariadb/data/kimconnect.sql
      mysql -uroot -p$rootPassword test < $sqlDump
      grantUser=bn_wordpress # this is the default Bitnami WordPress user
      echo "GRANT ALL PRIVILEGES ON $oldDb.* TO $grantUser;" | mysql -uroot -p$rootPassword
      #echo "create database $databaseName;" | mysql -uroot -p$rootPassword
      #mysql -uroot -p$rootPassword $oldDb -sNe 'show tables' | while read table; do mysql -uroot -p$rootPassword -sNe "RENAME TABLE $oldDb.$table TO $newDb.$table"; done
      #echo "create user kimconne_blog@localhost;grant all privileges on kimconne_blog.* to 'kimconne_blog';"| mysql -uroot -p$rootPassword
      #ALTER USER 'kimconne_blog'@'localhost' IDENTIFIED BY 'SOMEPASSWORDHERE';
  - Files:
    - Assuming nfs:
      nfsShare=k8s
      nfsServer=10.10.10.5
      sharePath=/volume1/$nfsShare
      mountPoint=/mnt/$nfsShare
      sudo mkdir $mountPoint
      sudo mount -t nfs $nfsServer:$sharePath $mountPoint # Test mounting
      sudo mount | grep $nfsShare # validate mount
      # Assuming Kubernetes NFS
      # sudo mv /home/$(whoami)/Downloads/localhost.sql $mountPoint/path_to_default-data-sitename-mariadb/data/localhost.sql
      # sudo mv /home/$(whoami)/Downloads/public_html.tar.bz2 $mountPoint/public_html.tar.bz2
      bz2File=/mnt/k8s/kimconnectblog/public_html.tar.bz2
      containerPath=/mnt/k8s/default-kimconnectblog-wordpress-pvc-9f1dd4bd-81f3-489f-9b76-bf70f4fd291c/wordpress/wp-content
      tar -xf $bz2File -C $containerPath
      cd $containerPath
      mv public_html/wp-content wp-content
      vim wp-config.php # edit wp config to match the imported database and its prefix

WordPress: Add Search Box Into Header

Navigate to Appearance > Theme Editor > select header.php > search for this section:

					<div id="site-header-menu" class="site-header-menu">
						<?php if ( has_nav_menu( 'primary' ) ) : ?>
							<nav id="site-navigation" class="main-navigation" role="navigation" aria-label="<?php esc_attr_e( 'Primary Menu', 'twentysixteen' ); ?>">
								<?php
									wp_nav_menu(
										array(
											'theme_location' => 'primary',
											'menu_class' => 'primary-menu',
										)
									);
								?>
							</nav><!-- .main-navigation -->

Insert this:

						<div class="header-search">
							<?php get_search_form(); ?>
						</div>

Optimizing Windows IIS Server

Most IIS instances are fine with default settings. However, it’s often beneficial to configure some recommended settings on multi-core machines so that Web Services would run optimally:

  1. Set CPU Limit to prevent server from maxing out CPU when there are ‘runaway’ processes. Here’s an example config on a 16-core server:
  2. Specify processor affinity to exclude IIS utilization of CPU-0. As the networking stack is highly dependent on this default core, reserving it for OS and packets processing would be improve resource handling and overall server processing speed. On NUMA enabled servers, this would apply be required as an extra step to enable processor affinity:
    # IIS 10.0+ would require disabling IIS Thread Pool Ideal CPU Optimization on a Non-Uniform Memory Access (NUMA) server
    $regKeyCpuOptimization='REGISTRY::HKLM\System\CurrentControlSet\Services\InetInfo\Parameters'
    $keyName='ThreadPoolUseIdealCpu'
    $enable=1
    $disable=0
    set-itemproperty -Path $regKeyCpuOptimization -Name $keyName -Value $disable
  3. Disable Overlapped Recycle should be set to True – as opposed to the default value of False. Recycling “Specifies whether the WWW Service should start another worker process to replace the existing worker process while that process is shutting down. The value of this property should be set to true if the worker process loads any application code that does not support multiple worker processes. (source: https://docs.microsoft.com/en-us/iis/configuration/system.applicationhost/applicationpools/add/recycling/)
  4. Cap Virtual Memory Limit to about 85% of server’s RAM capacity would ensure that  there is enough reserve for the OS and antivirus programs.
    $memoryPercent=85
    $memoryKb=(gwmi -Class win32_operatingsystem).TotalVisibleMemorySize
    write-host "$memoryPercent`% of RAM on $env:computername is $($memoryKb*$memoryPercent/100)"

PowerShell: How To Invoke Rest Method with RingCentral Rest API

[string]$restApiTokenUrl="https://platform.ringcentral.com/restapi/oauth/token"
[string]$restApiUrl="https://platform.ringcentral.com/restapi/v1.0/account/~/extension?page=1"
[string]$username='15555555555'
[string]$password='PASSWORDHERE'
[string]$extension='100'
[string]$appKey='APPKEYHERE'
# Part 1: Obtain Rest-API Token / Authorization
function obtainRingCentralTokenAuth{
  param(
      [string]$restApiTokenUrl,
      [string]$username,
      [string]$password,
      [string]$extension,
      [string]$appKey
      )
      [hashtable]$postAuthorization = @{
        grant_type = 'password'
        username = $username
        password = $password
        extension = $extension
      }
    $headers=New-Object "System.Collections.Generic.Dictionary[[String],[String]]"
    $headers.Add("Authorization", "Basic $appKey")
    $headers.Add("Content-Type", "application/x-www-form-urlencoded;charset=UTF-8")
    $token=Invoke-RestMethod -Method Post -Uri $restApiTokenUrl -Body $postAuthorization -headers $headers -ContentType "application/x-www-form-urlencoded"
    $authorization = $token.token_type + " " +$token.Access_token
    return $authorization
}

# Part 2:
function getRecordsViaRestApi{
    param(
        [string]$restApiUrl,
        [string]$authorization
    )
    try{
        $results=Invoke-RestMethod -uri $restApiUrl -Headers @{"Authorization"="$authorization"} -ea Stop
        return $results
    }catch{
        write-warning $_
        return $false    
    }
}

$authorization=obtainRingCentralTokenAuth $restApiTokenUrl $username $password $extension $appKey
getRecordsViaRestApi $restApiUrl $authorization

PowerShell: How To Set IP and Domain Restrictions to Specific IIS Sites

# Enable IP Filtering Feature in IIS using PowerShell
Install-WindowsFeature Web-IP-Security 
Restart-Service W3SVC

# Optional: Run IIS Manager GUI
# $env:windir\system32\inetsrv\InetMgr.exe

# Select the default website
$defaultWebsite=get-website|?{$_.id -eq 1}
$physicalPath=$defaultWebsite.PhysicalPath
$subSite=''
$entryPoint=$physicalPath+$subSite

# Show files
$siteFiles=gci $physicalPath
$siteFiles|write-host
$index=$siteFiles|?{$_.Name -match '^(default|index)\.\w+$'}
write-host "Index file is: $($index.FullName)"

# Deny All
Set-WebConfigurationProperty -Filter '/system.webServer/security/ipSecurity' -Location "$entryPoint" -Name allowUnlisted -Value False

# Deny from a specific IP or network
$ipAddress='192.168.20.0'
$subnetMask=24
$allowed='false'
Add-WebConfiguration -Filter '/system.webServer/security/ipSecurity' -Location "$entryPoint" -Value @{ipAddress="$ipAddress";subnetMask="$subnetMask";allowed="$allowed"}

# Allow traffic from a specific IP or network
$ipAddress='192.168.20.0'
$subnetMask=24
$allowed='true'
Add-WebConfiguration -Filter '/system.webServer/security/ipSecurity' -Location "$entryPoint" -Value @{ipAddress="$ipAddress";subnetMask="$subnetMask";allowed="$allowed"}

# Restart
Restart-WebItem -PSPath "IIS:\Sites\$entryPoint"

Kubernetes Container Deployment with NFS Persistent Volumes

Introduction:

Update: we have provided a practical application of the knowledge conveyed in this article with a new document on How to Deploy Pihole in Kubernetes here.

Now continuing with the contents of this blog…

One of the first questions for container storage provisioning is ‘why NFS? ‘ The answer to such inquiries is almost always ‘it depends.’ Here’s an overview of common storage protocols to determine the appropriate type for a Kubernetes system:

SMB/CIFS
– Abbreviation: SMB is Server Messaging Protocol/Common Internet File System
– Ports: 137/UDP  138/TCP  139/TCP 445/TCP
– Very chatty protocol – ideal for Windows environment
– More secured than NFS. Provides some security features
– Less scalable, ‘normal’ speed, and complex to setup (in Linux environments)

NFS
– Abbreviation: Network File System
– Ports: 111/TCP 111/UDP 2049/TCP 2049/UDP 1110/TCP 1110/UDP 4045/TCP 4045/UDP
– Less chatty – ideal for Linux environments
– Not a secured protocol – IP filtering is required as an access barrier as there are no username/password authentications
– Very scalable, fast, easy to setup

iSCSI
– Abbreviation: Internet Small Computer System Interface
– Ports: 860/TCP 860/UDP 3260/TCP 3260/UDP
– Less chatty – ideal for dedicated subnets for storage communcations
– Secured protocol – IP filtering as well as CHAP authentication (username/password)
– Less scalable, fast, more complex to setup (with networking knowledge necessary)

Step 0: Prepare Network File System (NFS) Share

There are many vendors of network storage appliances available on the market, many of which would support iSCSI, CIFS/SMB, NFS, FTP, SFTP, and even Rsync. Instructions on how to create a new NFS share would vary on each of those appliances.

In this example, we’re using OpenMediaVault (OVM), a derivative of FreeNAS with a difference in base OS of Debian Linux vs FreeBSD. Here’s a screenshot of a NFS share from within the OVM interface.

The important note here is that such a share would be set at the OS layer with Access Control List (ACL) of 750 (with nfs daemon as the owner) or 777 (world access) to enable read/write access. Moreover, NFS permissions would be RW,subtree_check,insecure with client access allowed from the subnet where the external IP of the Kubernetes cluster would ingress/egress. That would be the same network as the worker nodes (e.g. 192.168.80.0/24, an arbitrary private subnet to be configured in this lab for K8)

Step 1: Test the NFS Share Access On Each Node

# Install prerequisite on the Master node
sudo apt-get install nfs-common
 
# Set variables
nfsShare=test # assuming that the 'test' share has already been created on the server
nfsServer=NasServerNameOrIP
mountPoint=/mnt/test
sudo mkdir $mountPoint
sudo mount -t nfs $nfsServer:/$nfsShare $mountPoint

# Example of success
# Nothing, no feedback output from CLI

# Example of failure which will require fixing the file share on the NFS server
brucelee@controller:$ sudo mount -t nfs $nfsServer:/$nfsShare $mountPoint
mount.nfs: access denied by server while mounting FILESERVER:/test
# Create an index file to be used by NGINX - Do this only once
echo "NFS Persistent Volume Test in Kubernetes is Successful!" >> $mountPoint/index.html
cat $mountPoint/index.html

# Example of success
brucelee@controller:/mnt$ cat $mountPoint/index.html
NFS Persistent Volume Test in Kubernetes is Successful!

Step 2: Create Storage Class Name ‘nfs-class’

# Check storage classes - default installation of K8 will have no custom storage classes
kim@linux01:~$ kubectl get storageclasses
No resources found

# Create custom storage class - this will fail if nfs-class has already been manually created prior, which is a desired outcome.
storageClassName=nfs-class
cat > $storageClassName.yaml <<EOF
kind: StorageClass
apiVersion: storage.k8s.io/v1
metadata:
  name: $storageClassName
provisioner: kubernetes.io/nfs
reclaimPolicy: Retain
allowVolumeExpansion: true
EOF
kubectl apply -f $storageClassName.yaml

Step 3: Create a Persistent Volume

# Set variables
pvName=test-nfs-volume
storageClassName=nfs-class
storageSize=100Gi
nfsServer=192.168.80.80
nfsShare=test

# Create yammal file
cat > $pvName.yaml << EOF
apiVersion: v1
kind: PersistentVolume
metadata:
  name: $pvName
spec:
  storageClassName: $storageClassName
  capacity:
    storage: $storageSize 
  accessModes:
  - ReadWriteMany 
  nfs: 
    path: /$nfsShare 
    server: $nfsServer
  persistentVolumeReclaimPolicy: Retain # Other options: Recycle = rm -rf /test/* , Delete = eteled
EOF

# Apply the thing
kubectl apply -f $pvName.yaml

Step 4: Create a Persistent Volume Claim

pvClaimName=test-nfs-claim
storageClassName=nfs-class
claimSize=100Gi
cat > $pvClaimName.yaml << EOF
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: $pvClaimName
spec:
  storageClassName: $storageClassName
  accessModes:
  - ReadWriteMany
  resources:
    requests:
      storage: $claimSize
EOF
kubectl apply -f $pvClaimName.yaml

Step 5: Create Deployment Plan

# Set variables
deploymentName=test-nfs-deployment
replicas=2
appName=test
imageSource=nginx:alpine
containerPort=80
containerMountPath=/usr/share/nginx/html
pvName=test-nfs-volume
pvClaimName=test-nfs-claim

# Create deployment file
cat > $deploymentName.yaml << EOF
kind: Deployment
apiVersion: apps/v1
metadata:
  name: $deploymentName
spec:
  replicas: $replicas
  selector: 
    matchLabels:
      app: $appName # This must be identical to the pod name (template label)
  strategy:
    type: RollingUpdate
    rollingUpdate:
      maxSurge: 1
      maxUnavailable: 1    
  template:    
    metadata:
      labels:
        app: $appName
    spec:
      hostNetwork: true # This allows a direct ingress to any node. When this value is set, the container must not be binding to ports that are in use by the worker nodes (e.g. 53/tcp 53/udp for dns)
      containers:
      - name: $appName
        image: $imageSource
        ports:
          - containerPort: $containerPort
            name: $appName
        volumeMounts:
          - mountPath: $containerMountPath
            name: $pvName # this must matches volume name
      volumes:
        - name: $pvName
          persistentVolumeClaim:
            claimName: $pvClaimName
EOF

# Apply deployment plan
kubectl apply -f $deploymentName.yaml

Step 6: Implement MetalLB Load Balancer

# Set strictARP, ipvs mode
kubectl get configmap kube-proxy -n kube-system -o yaml | \
sed -e "s/strictARP: false/strictARP: true/" | sed -e "s/mode: \"\"/mode: \"ipvs\"/" | \
kubectl apply -f - -n kube-system
 
# Apply the manifests provided by the author, David Anderson (https://www.dave.tf/) - an awesome dude
kubectl apply -f https://raw.githubusercontent.com/metallb/metallb/v0.9.5/manifests/namespace.yaml
kubectl apply -f https://raw.githubusercontent.com/metallb/metallb/v0.9.5/manifests/metallb.yaml
# On first install only
kubectl create secret generic -n metallb-system memberlist --from-literal=secretkey="$(openssl rand -base64 128)"
 
# Sample output:
brucelee@controller:~$ kubectl apply -f https://raw.githubusercontent.com/metallb/metallb/v0.9.5/manifests/namespace.yaml
namespace/metallb-system created
brucelee@controller:~$ kubectl apply -f https://raw.githubusercontent.com/metallb/metallb/v0.9.5/manifests/metallb.yaml
podsecuritypolicy.policy/controller created
podsecuritypolicy.policy/speaker created
serviceaccount/controller created
serviceaccount/speaker created
clusterrole.rbac.authorization.k8s.io/metallb-system:controller created
clusterrole.rbac.authorization.k8s.io/metallb-system:speaker created
role.rbac.authorization.k8s.io/config-watcher created
role.rbac.authorization.k8s.io/pod-lister created
clusterrolebinding.rbac.authorization.k8s.io/metallb-system:controller created
clusterrolebinding.rbac.authorization.k8s.io/metallb-system:speaker created
rolebinding.rbac.authorization.k8s.io/config-watcher created
rolebinding.rbac.authorization.k8s.io/pod-lister created
daemonset.apps/speaker created
deployment.apps/controller created
brucelee@controller:~$ kubectl create secret generic -n metallb-system memberlist --from-literal=secretkey="$(openssl rand -base64 128)"
secret/memberlist created
 
# Customize for this system
ipRange=192.168.1.80-192.168.1.89
loadBalancerFile=metallb-config.yaml
cat > $loadBalancerFile << EOF
apiVersion: v1
kind: ConfigMap
metadata:
  namespace: metallb-system
  name: config
data:
  config: |
    address-pools:
    - name: default
      protocol: layer2
      addresses:
      - $ipRange
EOF
kubectl apply -f $loadBalancerFile
 
# Sample output
brucelee@controller:~$ kubectl apply -f $fileName
configmap/config created

Step 7: Create a Service Cluster

serviceName=test-service
appName=test
nodePort=30000
containerPort=80
servicePort=80
cat > test-service.yaml << EOF
apiVersion: v1
kind: Service
metadata:
  name: $serviceName
spec:
  type: LoadBalancer # Other options: ClusterIP, LoadBalancer
  selector:
    app: $appName # This name must match the template.metadata.labels.app value
  ports:
  - protocol: TCP
    port: $servicePort
    targetPort: $containerPort
    # nodePort: $nodePort # optional field: by default, Kubernetes control plane will allocate a port from 30000-32767 range
EOF
kubectl apply -f test-service.yaml
clusterIP=$(kubectl get service test-service --output yaml|grep 'clusterIP: '|awk '{print $2}')
echo "clusterIP: $clusterIP"
curl $clusterIP
kubectl get service test-service

Troubleshooting

A) Pod stuck in ContainerCreating status

brucelee@controller:~$ k get pod
NAME                                   READY   STATUS              RESTARTS   AGE
test-nfs-deployment-54b78bc4c6-4pdz8   0/1     ContainerCreating   0          86s
test-nfs-deployment-54b78bc4c6-sgbw8   0/1     ContainerCreating   0          86s

brucelee@controller:~$ kubectl describe pods
--- Truncated for brevity ---
Events:
  Type     Reason       Age                From               Message
  ----     ------       ----               ----               -------
  Normal   Scheduled    4m9s               default-scheduler  Successfully assigned default/test-nfs-deployment-54b78bc4c6-sgbw8 to linux03
  Warning  FailedMount  2m6s               kubelet            Unable to attach or mount volumes: unmounted volumes=[test-nfs-volume], unattached volumes=[test-nfs-volume default-token-bdhxv]: timed out waiting for the condition
  Warning  FailedMount  2m (x9 over 4m8s)  kubelet            MountVolume.SetUp failed for volume "test-nfs-volume" : mount failed: exit status 32
Mounting command: mount
Mounting arguments: -t nfs 192.168.100.21:/test /var/lib/kubelet/pods/8aa113c6-1b1e-4329-ad37-f9f04fd72e78/volumes/kubernetes.io~nfs/test-nfs-volume
Output: mount: /var/lib/kubelet/pods/8aa113c6-1b1e-4329-ad37-f9f04fd72e78/volumes/kubernetes.io~nfs/test-nfs-volume: bad option; for several filesystems (e.g. nfs, cifs) you might need a /sbin/mount.<type> helper program.

Resolution:
- Check on the NFS server share to ensure that its been set with RW,insecure and such folder has been set with at least 750 permissions (777 preferred)
- Check the Pod Deployment template:spec:hostNetwork: true has been set
- Run this on each node: sudo apt-get install nfs-common

B) Error when the name of spec:containers:volumeMounts.name doesn't match spec:volumes:name

The Deployment is invalid: spec.template.spec.containers[0].volumeMounts[0].name: Not found:

C) Error when no storage class 'nfs-class' has NOT been defined

brucelee@controller:~$ k describe persistentvolumeclaims
Name:          test-nfs-claim
Namespace:     default
StorageClass:  nfs
Status:        Pending
Volume:        
Labels:        <none>
Annotations:   <none>
Finalizers:    [kubernetes.io/pvc-protection]
Capacity:      
Access Modes:  
VolumeMode:    Filesystem
Used By:       test-nfs-deployment-6d4bff899f-5t2m2
               test-nfs-deployment-6d4bff899f-kds6l
Events:
  Type     Reason              Age                      From                         Message
  ----     ------              ----                     ----                         -------
  Warning  ProvisioningFailed  4m21s (x763 over 3h14m)  persistentvolume-controller  storageclass.storage.k8s.io "nfs" not found

Step 8: Cleanup

# Cleanup: must be in the correct sequence!
kubectl delete services test-service
kubectl delete deployment test-nfs-deployment
kubectl delete persistentvolumeclaims test-nfs-claim
kubectl delete pv test-nfs-volume

Java Virtual Machine Optimal Memory Tuning

Overview:

There are five available garbage collectors (GC) for Java Virtual Machines (JVM). Here are some quick lesions-learned on each GC engines:

  1. G1: is the default for Java 9 and newer. It’s the best choice for real-time applications with rapid vertical scaling. Some tests show that this is slower than Parallel GC, although Parallel is known to oversubscribe allotted RAM limits leading to application slowness.
  2. Parallel: is the default for Java 8 or older. this engine does everything at once which will result in random lags. It’s intended for Applications where throughput is the focus not real-time usage.
  3. ConcMarkSweep (CMS): is designed to eliminate the long pause associated with the full gc of parallel & serial collector. It’s similar to G1 by using multiple background threads to scan and clear heaps
  4. Serial: good for single virtual CPU machines. Nobody uses this GC nowadays
  5. Shenandoah: is available on JDK 12, a super low-latency GC that operates mostly concurrently with the application. It’s most apppriate for gambling, finance, and latency-sensitive interactive apps. This engine costs more CPU consumption than Parallel GC.
Practical Examples:

Similar to the process of SQL server performance tuning, Java memory allocation should be manually configured according to the host’s RAM availability to ensure machine up-time with performance consistency. Bottom line is that 4G should be allocated to JVM for servers with 12 GB or less, or 80% of available memory for servers with more than 12 GB. The most important points are setting explicit Garbage collection as G1, specifying low memory heap, adding a high reserve, etc.

12GB of RAM:

java.args=-server -Xms8G -Xmx8G -XX:+UseG1GC -XX:+ParallelRefProcEnabled -XX:MaxGCPauseMillis=200 -XX:+UnlockExperimentalVMOptions -XX:+DisableExplicitGC -XX:+AlwaysPreTouch -XX:G1NewSizePercent=30 -XX:G1MaxNewSizePercent=40 -XX:G1HeapRegionSize=8M -XX:G1ReservePercent=20 -XX:G1HeapWastePercent=5 -XX:G1MixedGCCountTarget=4 -XX:InitiatingHeapOccupancyPercent=15 -XX:G1MixedGCLiveThresholdPercent=90 -XX:G1RSetUpdatingPauseTimePercent=5 -XX:SurvivorRatio=32 -XX:+PerfDisableSharedMem -XX:MaxTenuringThreshold=1 nogui ... OTHER ARGS ...

16GB of RAM:

java.args=-server -Xms13107m -Xmx13107m -XX:+UseG1GC -XX:+ParallelRefProcEnabled -XX:MaxGCPauseMillis=200 -XX:+UnlockExperimentalVMOptions -XX:+DisableExplicitGC -XX:+AlwaysPreTouch -XX:G1NewSizePercent=40 -XX:G1MaxNewSizePercent=50 -XX:G1HeapRegionSize=16M -XX:G1ReservePercent=15 -XX:G1HeapWastePercent=5 -XX:G1MixedGCCountTarget=4 -XX:InitiatingHeapOccupancyPercent=20 -XX:G1MixedGCLiveThresholdPercent=90 -XX:G1RSetUpdatingPauseTimePercent=5 -XX:SurvivorRatio=32 -XX:+PerfDisableSharedMem -XX:MaxTenuringThreshold=1 nogui ... OTHER ARGS ...

12GB of RAM with ColdFusion, IIS, Windows Server 2016:

java.args=-server -Xms8G -Xmx8G -XX:+UseG1GC -XX:+ParallelRefProcEnabled -XX:MaxGCPauseMillis=200 -XX:+UnlockExperimentalVMOptions -XX:+DisableExplicitGC -XX:+AlwaysPreTouch -XX:G1NewSizePercent=30 -XX:G1MaxNewSizePercent=40 -XX:G1HeapRegionSize=8M -XX:G1ReservePercent=20 -XX:G1HeapWastePercent=5 -XX:G1MixedGCCountTarget=4 -XX:InitiatingHeapOccupancyPercent=15 -XX:G1MixedGCLiveThresholdPercent=90 -XX:G1RSetUpdatingPauseTimePercent=5 -XX:SurvivorRatio=32 -XX:+PerfDisableSharedMem -XX:MaxTenuringThreshold=1 --add-opens=java.rmi/sun.rmi.transport=ALL-UNNAMED --add-opens=java.base/java.nio=ALL-UNNAMED --add-opens=java.base/java.lang=ALL-UNNAMED --add-opens=java.base/sun.util.cldr=ALL-UNNAMED --add-opens=java.base/sun.util.locale.provider=ALL-UNNAMED -Xbatch -Djdk.attach.allowAttachSelf=true -Dcoldfusion.home={application.home} -Duser.language=en -Dcoldfusion.rootDir={application.home} -Dcom.sun.xml.bind.v2.bytecode.ClassTailor.noOptimize=true -Dcoldfusion.libPath={application.home}/lib -Dorg.apache.coyote.USE_CUSTOM_STATUS_MSG_IN_HEADER=true -Dcoldfusion.jsafe.defaultalgo=FIPS186Random -Dorg.eclipse.jetty.util.log.class=org.eclipse.jetty.util.log.JavaUtilLog -Djava.util.logging.config.file={application.home}/lib/logging.properties -Djava.locale.providers=COMPAT,SPI -Dsun.font.layoutengine=icu -Dcoldfusion.classPath={application.home}/lib/updates,{application.home}/lib,{application.home}/lib/axis2,{application.home}/gateway/lib/,{application.home}/wwwroot/WEB-INF/cfform/jars,{application.home}/wwwroot/WEB-INF/flex/jars,{application.home}/lib/oosdk/lib,{application.home}/lib/oosdk/classes

Explanations:

-Xms: sets the starting global memory heap size to prevent pauses caused by heap expansion
-Xmx: places upper boundary on the global heap size to increase the predictability of garbage collection
-XX:+UseG1GC: use the Garbage First (G1) Collector, instead of relying on Explicit GC. The Garbage-First (G1) collector is a server-style garbage collector, targeted for multi-processor machines with large memories. It meets garbage collection (GC) pause time goals with a high probability, while achieving high throughput. The G1 garbage collector is fully supported in Oracle JDK 7 update 4 and later releases. (Source: https://www.oracle.com/technetwork/tutorials/tutorials-1876574.html)
-XX:MaxGCPauseMillis: sets a target for the maximum GC pause time so the engine would use as baseline
-XX:+ParallelRefProcEnabled: multi-thread reference processing, reducing young and old GC times
-XX:MaxGCPauseMillis: sets the peak pause time expected in the environment. 250 ms as the default is adequate for most systems. When this value is set lower than 200, it causes GC to run more aggressively and less efficiently, which can steal cycles without yielding considerable benefit
-XX:+UnlockExperimentalVMOptions: required to activate experiemental parameters
-XX:+AlwaysPreTouch option the JVM touches every single byte of the max heap size with a '0', resulting in the memory being allocated in the physical memory in addition to being reserved in the internal data structure (virtual memory). Pretouching is single threaded, so it is expected behavior that it causes JVM startup to be delayed. The trade off is that it will reduce page access time later, as the pages will already be loaded into memory. (Source: https://access.redhat.com/solutions/2685771)
-XX:+DisableExplicitGC: in conjunction with -XX:+UseG1GC to force JVM to use G1GC
-XX:ParallelGCThreads: controls the parallelism of global GC phases, which should include parallel reference processing
-XX:ConcGCThreads: number of threads for garbage collectors
-XX:InitiatingHeapOccupancyPercent: percentage of the global heap size as trigger to start a concurrent GC cycle. Please note that a value of 0 denotes 'constant GC cycles', and the default value is 45
-XX:G1NewSizePercent: Sets the percentage of the heap to use as the minimum for the young generation size. The default value is 5 percent
-XX:G1MaxNewSizePercent: percentage of the heap size to use as the maximum for young generation size. The default value is 60 percent
-XX:G1HeapRegionSize: reduce fragmentation of old generation by setting this value higher
-XX:G1ReservePercent: option to increase the amount of reserve memory for next spaces
-XX:G1HeapWastePercent: percentage of heap that you are willing to allow as non-deallocation
-XX:G1MixedGCCountTarget: sets the target number of mixed garbage collections after a marking cycle to collect old regions with at most G1MixedGCLIveThresholdPercent live data
-XX:G1MixedGCLiveThresholdPercent (source: https://www.oracle.com/technical-resources/articles/java/g1gc.html)
-XX:G1RSetUpdatingPauseTimePercent: percent of the allowed maximum pause time
-XX:SurvivorRatio: controls the size of the survivor spaces. During 'young' spaces collection, every single object is copied. The Object may be copied to one of survival spaces. For each object being copied, GC algorithm increments its age (aka number of collection survived). If the age is above the current tenuring threshold, it would be copied to what is known as an 'old' space. The Object could also be copied to the old space directly if the survival space gets full - this is called an 'overflow.' If value is set too low, collection copying will overflow into the old generation. If this value is too high, some spaces will be empty.The default value should be 32 as that is known to keep the spaces half-filled.
-XX:+PerfDisableSharedMem: feature to reduce worst-case pause latencies
-XX:MaxTenuringThreshold: specifies for how many minor GC cycles an object will stay in the survivor spaces until it finally gets tenured into the old space

 

How to Host Multiple Domains with SSL Using Microsoft Information Service (IIS)

Step 1: Add SSL Certs into the computer information store

Here’s a sample script to import a wildcard SSL cert while login via console (e.g. mstsc /admin):

# Import Cert and get its thumbprint
$certPath="C:\wildcard_kimconnectcom.pfx"
$certPlaintextPassword='PASSWORDHERE'
$certEncryptedPassword=ConvertTo-SecureString $certPlaintextPassword -AsPlainText -Force
$importedCert=Import-PfxCertificate -CertStoreLocation Cert:\LocalMachine\My -FilePath $certPath -Password $certEncryptedPassword
$importedCertThumbprint=$importedCert.Thumbprint

Alternatively, SSL certs may be added to the ‘Web Hosting’ store, instead of the ‘Personal’ store. Per Microsoft, ‘The Web Hosting store works like the Personal store, so all of the existing tools to import and export certificates work the same way. The key difference between Web Hosting store and Personal store is that Web Hosting store is designed to scale to higher numbers of certificates.’ In practice, administrators have a habit of searching for certificates inside the ‘Personal’ store of the local machine. Hence, items inside the Web Hosting store may be missed by unaware admins. Thus, it’s considered a standard practice to ignore this alternative unless the target web server is ultra high density.

There’s the other option to store site certs within IIS ‘Server Certificates’ container. Such feature would automatically pull local machine certs from the usual locations of ‘Personal’ and ‘Web Hosting’, along with capacity to place them in a separate and customized folder path on the server. This offers additional flexibility with the trade off in terms of management simplicity. IIS Server Certificates installation and configuration is outside the scope of this article.

Step 2: Bind SSL Certs to Multiple Sites
Option A: Configure server with multiple IPs

Traditionally, a network end-point can only be identified with one-to-one IP:Port binding. Therefore, this configuration would be straight-forward. Here is a demonstration:

  • Overload one IP per domain to be hosted on the web server as per this example:
    • Configure server private IP 10.10.10.11 for kimconnect.com 
    • Configure server private IP 10.10.10.12 for kimconnect2.com
  • Create a NAT policy to forward one public IP toward one private IP of the web server. Here’s an example for 2 domains:
    • Public IP 1.1.1.1 is mapped to 10.10.10.11
    • Public IP 1.1.1.2 is mapped to 10.10.10.12
  • Bind each site to a separate private IP
    Assuming that IIS has already been installed on the target Windows Server, run inetmgr.exe > navigate to [Server Name] node > Sites > add two (2) websites with names of kimconnect.com and kimconnect2.com > right-click each site > Edit bindings > Add or Edit bindings to resemble these:
    • Bind kimconnect.com toward 10.10.10.11 as shown below
    • Bind kimconnect2.com toward 10.10.10.12 as illustrated
Option B: Use Server Name Indication (SNI) to host multiple sites

Starting with Windows Server 2012, where the certificate store and SNI are part of default IIS installation. There are no specific IIS features that need to be installed from Server Manager. Here is another demonstration to configure IIS 8.0 or higher to use a single IP to host multiple sites with different domain SSL certs.

  • On the web server, add records for each domain to be hosted:
    • Edit C:\windows\system32\drivers\etc\hosts using Notepad as Administrator
    • Add these sample entries:
      127.0.0.1 kimconnect.com
      127.0.0.1 kimconnect2.com
  • Add sites with SNI bindings
    One must be mindful not combine traditional SSL bindings (IP:Port) and SNI bindings (Hostname:Port) configurations on the same server. Hence, it is recommended that legacy binding methods be excluded in this approach.
    • Bind kimconnect.com toward ‘*’ or ‘All Unassigned’ as shown below
    • Bind kimconnect2.com toward ‘*’ or ‘All Unassigned’ as shown below
  • Verify that the sites are using different SSL certificate hashes
    PS C:\Users\TestAdmin> netsh http show sslcert

    SSL Certificate bindings:
    -------------------------

    Hostname:port : kimconnect.com:443
    Certificate Hash : 12345abcde
    Application ID : {something-something}
    Certificate Store Name : My
    Verify Client Certificate Revocation : Enabled
    Verify Revocation Using Cached Client Certificate Only : Disabled
    Usage Check : Enabled
    Revocation Freshness Time : 0
    URL Retrieval Timeout : 0
    Ctl Identifier : (null)
    Ctl Store Name : (null)
    DS Mapper Usage : Disabled
    Negotiate Client Certificate : Disabled
    Reject Connections : Disabled
    Disable HTTP2 : Not Set

    Hostname:port : kimconnect2.com:443
    Certificate Hash : 67890fghijk
    Application ID : {something2-something2}
    Certificate Store Name : My
    Verify Client Certificate Revocation : Enabled
    Verify Revocation Using Cached Client Certificate Only : Disabled
    Usage Check : Enabled
    Revocation Freshness Time : 0
    URL Retrieval Timeout : 0
    Ctl Identifier : (null)
    Ctl Store Name : (null)
    DS Mapper Usage : Disabled
    Negotiate Client Certificate : Disabled
    Reject Connections : Disabled
    Disable HTTP2 : Not Set

Linux: How to Display the SSL Certificate of a Remote Server URL

Command:

server=test.kimconnect.com
echo | openssl s_client -showcerts -servername $server -connect $server:443 2>/dev/null | openssl x509 -inform pem -noout -text
kim@kimlinux:~$ echo | openssl s_client -showcerts -servername test.kimconnect.com -connect test.kimconnect.com:443 2>/dev/null | openssl x509 -inform pem -noout -text
Certificate:
    Data:
        Version: 3 (0x2)
        Serial Number:
        Signature Algorithm: ecdsa-with-SHA256
        Issuer: C = US, O = "Cloudflare, Inc.", CN = Cloudflare Inc ECC CA-3
        Validity
            Not Before: Aug 15 00:00:00 2020 GMT
            Not After : Aug 15 12:00:00 2021 GMT
        Subject: C = US, ST = CA, L = San Francisco, O = "Cloudflare, Inc.", CN = sni.cloudflaressl.com
        Subject Public Key Info:
            Public Key Algorithm: id-ecPublicKey
                Public-Key: (256 bit)
                pub:
                ASN1 OID: prime256v1
                NIST CURVE: P-256
        X509v3 extensions:
            X509v3 Authority Key Identifier: 
                keyid:
            X509v3 Subject Key Identifier: 
            X509v3 Subject Alternative Name: 
                DNS:sni.cloudflaressl.com, DNS:*.kimconnect.com, DNS:kimconnect.com
            X509v3 Key Usage: critical
                Digital Signature
            X509v3 Extended Key Usage: 
                TLS Web Server Authentication, TLS Web Client Authentication
            X509v3 CRL Distribution Points: 
                Full Name:
                  URI:http://crl3.digicert.com/CloudflareIncECCCA-3.crl
                Full Name:
                  URI:http://crl4.digicert.com/CloudflareIncECCCA-3.crl
            X509v3 Certificate Policies: 
                Policy: 2.16.840.1.114412.1.1
                  CPS: https://www.digicert.com/CPS
                Policy: 2.23.140.1.2.2
            Authority Information Access: 
                OCSP - URI:http://ocsp.digicert.com
                CA Issuers - URI:http://cacerts.digicert.com/CloudflareIncECCCA-3.crt
            X509v3 Basic Constraints: critical
                CA:FALSE
            CT Precertificate SCTs: 
                Signed Certificate Timestamp:
                    Version   : v1 (0x0)
                    Log ID    : 
                    Timestamp : Aug 15 10:43:23.137 2020 GMT
                    Extensions: none
                    Signature : ecdsa-with-SHA256
                Signed Certificate Timestamp:
                    Version   : v1 (0x0)
                    Log ID    : 
                    Timestamp : Aug 15 10:43:23.189 2020 GMT
                    Extensions: none
                    Signature : ecdsa-with-SHA256                                
    Signature Algorithm: ecdsa-with-SHA256

PowerShell: Start IIS Sites & App Pools

Before running the below script, it’s necessary to mark certain IIS App pools and websites to automatically start. Hence, those services are expected to be up at all times. Here’s are some some quick instructions on how to enable that:

Run InetMgr.exe > Navigate to Application Pools > right-click an App Pool item > Advanced Settings…

Set Start Mode = ‘AlwaysRunning’ > OK

$servers='IRV-WEBSERVER05'
$results=[PSCustomObject]@{}
Invoke-Command -ComputerName $Servers {
    Import-Module -Name WebAdministration
    $sites=Get-Website|Where-Object serverAutoStart -eq $true
    foreach ($site in $sites) {
        switch ($site) {
            {(Get-WebAppPoolState -Name $_.applicationPool).Value -eq 'Stopped'} {
                write-host "$($_.applicationPool) is stopped. Restarting it..."
                try{
                    Start-WebAppPool -Name $_.applicationPool -ea Stop
                    $errors='none'
                }catch{
                    $errors=$_
                    Write-Warning $errors
                    }
                }finally{
                    $results+=@{
                        computername=$env:computername
                        item=$_.applicationPool
                        errorMessage=$errors
                        }
                }
            {$_.State -eq 'Stopped'} {
                write-host "$($site.Name) is stopped. Restarting it..."
                try{
                    Start-Website -Name $site.Name -ea Stop
                    $errors='none'
                }catch{
                    $errors=$_
                    Write-Warning $errors
                }finally{
                    $results+=@{
                        computername=$env:computername
                        item=$site.Name
                        errorMessage=$errors
                        }
                }              
            }
        }
    }
}

write-host ($results|out-string).trim()

IIS Error Code 0x80070021

Error Message:
Detailed Error Information:
Module IIS Web Core
Notification BeginRequest
Handler Not yet determined
Error Code 0x80070021
Config Error This configuration section cannot be used at this path. This happens when the section is locked at a parent level. Locking is either by default (overrideModeDefault="Deny"), or set explicitly by a location tag with overrideMode="Deny" or the legacy allowOverride="false".
Config File \\?\C:\HooliApp\web.config
Resolution:

Run InetMgr.exe > Sites > select Hooli Support Portal > right-click Configuration Editor > Set Application Pool, Physical Path, and authentication as Pass-through > Test settings > OK

Resolving LDAPS Connection Errors

Errors:
Can not connect to remote server: 5059 ERROR_CERTIFICATE_ERROR (unable to read server certificates from host=TESTDC.KIMCONNECT.com, port=636 error: java.net.SocketException: Connection reset) fields: [unable to read server certificates from host=TESTDC.KIMCONNECT.com, port=636 error: java.net.SocketException: Connection reset]
error connecting to ldap directory (default), error: unable to create connection: unable to bind to ldaps://TESTDC.KIMCONNECT.COM:636 as CN=test proxy,OU=Service Accounts,DC=kimconnect,DC=com reason: LDAP response read timed out, timeout used:30000ms.
Resolution

Install Active Directory certificates with these application policies:

  1. Client Authentication
  2. Server Authentication.

Please note that such services require Active Directory Lightweight Service (AD LS) and Active Directory Certificate Services (AD LS) Certification Authority (CA). On a member server, one would need install both components. Whereas, Domain Controllers already have the former available by means of being full-pledged DCs. This quick walk-through goes over both items.

Active Directory Lightweight Service
%systemroot%\ADAM\adaminstall.exe answerFile.txt

# Contents of answerFile.txt
[ADAMInstall]
InstallType=Unique
InstanceName=InstanceName
LocalLDAPPortToListenOn=PortNumber
LocalSSLPortToListenOn=PortNumber
NewApplicationPartitionToCreate=PartitionName
DataFilesPath=c:\ADAMInstances\InstanceName\Data
LogFilesPath=c:\ADAMInstances\InstanceName\Data
ServiceAccount=DomainorMachineName\AccountName
ServicePassword=Password
Administrator=Domain\GroupName
ImportLDIFFiles="LDIFFilename1" "LDIFFilename2" "LDIFFilename3"
SourceUserName=DomainorMachineName\AccountName SourcePassword=Password"
Active Directory Certificate Services:

Please be advised that the following instructions are assuming that the business requirement is to install a new Certification Authority on a Windows Server. There are cases where Subordinate Certification Authority would be more desirable. Please review this document prior to proceeding.

PowerShell Method

Install-WindowsFeature Adcs-Cert-Authority -IncludeManagementTools
Install-AdcsCertificationAuthority -CAType EnterpriseRootCa -CryptoProviderName "RSA#Microsoft Software Key Storage Provider" -KeyLength 2048 -HashAlgorithmName SHA256 -ValidityPeriod Years -ValidityPeriodUnits 10
# Sample Output:
PS C:\Windows\system32> Install-WindowsFeature Adcs-Cert-Authority -IncludeManagementTools
Success Restart Needed Exit Code Feature Result
------- -------------- --------- --------------
True No Success {Active Directory Certificate Services Too...

PS C:\Windows\system32> Install-AdcsCertificationAuthority -CAType EnterpriseRootCa -CryptoProviderName "RSA#Microsoft Software Key Storage Provider" -KeyLength 2048 -HashAlgorithmName SHA256 -ValidityPeriod Years -ValidityPeriodUnits 10

Confirm
Are you sure you want to perform this action?
Performing the operation "Install-AdcsCertificationAuthority" on target "TESTDC".
[Y] Yes [A] Yes to All [N] No [L] No to All [S] Suspend [?] Help (default is "Y"):

ErrorId ErrorString
------- -----------
0

PS C:\Windows\system32> certutil -verifystore MY
MY "Personal"
================ Certificate 0 ================
Serial Number: chicken4f86achicken42a0efchickenb
Issuer: CN=TESTDC-CA, DC=kimconnect, DC=com
NotBefore: 8/13/2010 9:42 PM
NotAfter: 8/13/2010 9:51 PM
Subject: CN=TESTDC-CA, DC=kimconnect, DC=com
CA Version: V0.0
Signature matches Public Key
Root Certificate: Subject matches Issuer
Cert Hash(sha1): chicken54988b0a23a757chickena4a13d5a8
Key Container = TESTDC-CA
Unique container name: abc123_chickenbob
Provider = Microsoft Software Key Storage Provider
Signature test passed
Verified Issuance Policies: All
Verified Application Policies: All
Certificate is valid
CertUtil: -verifystore command completed successfully.
# Verify presence of key container
PS C:\Windows\system32> ls C:\ProgramData\Microsoft\Crypto\Keys\

Directory: C:\ProgramData\Microsoft\Crypto\Keys

Mode LastWriteTime Length Name
---- ------------- ------ ----
-a--s- 8/13/2010 9:52 PM 1567 abc123_chickenbob

GUI Method

Run Server Manager > Manage > Add Roles and Features > select ‘Role-based or feature based installation > Next >  Next > put a check mark next to ‘Active Directory Certificate Services > Add Features > Next > Next > Next

Mark ‘Certification Authority’ > Next > Install

After AD CS role has been added > run AD CS Configuration wizard > input a Local Administrator credential > Next > select ‘Certification Authority’ > Next > select ‘Enterprise CA’ > Next > Choose ‘Root CA’ > Next > accept option ‘Create a new private key’ > Next > pick ‘RSA#Microsoft Software Key Storage Provider’ key length 2048 with hashing algorithm of SHA256 (or higher security if desired) > Next > input the ‘Common name’ to match the server name, enter the value of the ‘distinguished name suffix’ > Next > change value to ’10’ years > Next > Next > Configure > wait for completion > close

To verify, Run Certlm.msc > Personal > Certificates

PowerShell: Find a Downloadable File from a Web Page

Although the function below is adequate for simple purposes. It lacks the advanced algorithm to craw websites. I’ll loopback to rewrite this thing when there’s a necessity for better tools.

# findDownloadUrl_v0.0.2.ps1
# This little program will parse a web page for it's downloadable contents
# The multi-threading code has been added

$startUrl='https :// somewebsite'
$fileExtension='.exe'
$maxDepth=3

function findDownloadUrl{
    param(
        $startUrl,
        $fileExtension,
        $maxDepth=3
    )
    $simultaneousJobs=8
    $linksChecked=0
    $firstResult=$false
    $timer=[System.Diagnostics.Stopwatch]::StartNew()
    if(!$startUrl){
        write-warning "Cannot start with a blank parent URL"
    }elseif($startUrl -notmatch '/$'){
        $startUrl=$startUrl+'/'
        }

    function findFile($parentUrl,$extension){
        $ProgressPreference='SilentlyContinue'
        $ErrorActionPreference='stop'
        if($parentUrl -notmatch '/$'){$parentUrl=$parentUrl+'/'}
        try{
            $page=Invoke-WebRequest $parentUrl -TimeoutSec 10
        }catch{
            return @{'result'=$false;'links'=@()}
            }
        $newLinks=$page.links.href|?{$_ -notlike "*$(Split-Path $parentUrl -parent)"}| `
            sort -Descending|%{$(
                                if($_[0] -eq '/'){
                                    $parentUrl+$_.Substring(1,$_.length-1)
                                }elseif($_ -match '^http'){
                                    $_
                                }else{
                                    $parentUrl+$_
                                }
                            )}|select -Unique
        $matchedExtension=$newLinks|?{$_ -like "*$extension"}|sort -Descending|select -First 1
        if($matchedExtension){
            return @{'result'=$true;'links'=$matchedExtension}
        }elseif($newLinks){
            return @{'result'=$false;'links'=$newLinks}
        }else{
            return @{'result'=$false;'links'=@()}
            } 
    }  

    write-host "Scanning $startUrl for file extension $fileExtension"
    $startLinks=.{$result=findFile $startUrl $fileExtension
                    return $result['links']
                    }
    if($startLinks -eq [string]){
        return $startLinks
        }
    $knownLinks=$startLinks

    foreach ($link in $startLinks){       
        $currentDepth=1
        write-host "Processing link at current depth: $currentDepth"
        $newLinks=@($link) 
        do{ 
            if($i++ -lt $simultaneousJobs -and !(!$newLinks)){
                $thisLink=$newLinks|Select -Unique|select -First 1
                if($newLinks.count -gt 1){
                    $newLinks=$newLinks[1..($newLinks.count-1)]
                }else{
                    $newLinks=@()
                    }
                write-host "Parsing $thisLink"
                $job=start-job -ScriptBlock{
                    param($findFile,$thisLink,$fileExtension)
                    return [ScriptBlock]::Create($findFile).invoke($thisLink,$fileExtension)
                    } -Args ${function:findFile},$thisLink,$fileExtension
                $linksChecked++
            }else{
                do{
                    $results=Get-Job|Receive-Job -wait
                    get-job -State 'Completed'|remove-job                    
                    $results|%{
                        $currentDepth++
                        if($_['result']){
                            write-host "Bingo!" -ForegroundColor Green
                            get-job|remove-job
                            $firstResult=$_['links']
                        }elseif($currentDepth -le $maxDepth){
                            $addLinks=$_['links']|?{$_ -notin $knownLinks}
                            if($addLinks){
                                write-host "Adding new links to depth $currentDepth`:`r`n$(($addLinks|out-string).trim())"
                                $knownLinks+=$addLinks
                                $newLinks=$addLinks+$newLinks
                                }
                            }
                        }
                    $i=(get-job -state 'Running').count
                    }until($i -lt $simultaneousJobs -or $firstResult) 
                }
        }until((!$newLinks -and !$i) -or $firstResult)            
               
        if($firstResult){
            $totalMinutes=[math]::round($timer.Elapsed.TotalMinutes,2)
            write-host "Minutes elapsed: $totalMinutes"
            return $firstResult
            }
    }

    $totalMinutes=[math]::round($timer.Elapsed.TotalMinutes,2)
    write-host "$linksChecked links have been checked in $totalMinutes mintues without finding file extension $fileExtension" -ForegroundColor Red
    return $false
}
findDownloadUrl $startUrl $fileExtension $maxDepth
PS C:\Users\concu> findDownloadUrl $startUrl $fileExtension $maxDepth
Scanning http://apache.mirrors.pair.com/tomcat/tomcat-9/ for file extension .exe
Processing link at current depth: 1
Parsing http://apache.mirrors.pair.com/tomcat/tomcat-9/v9.0.37/
Adding new links to depth 2:
http://apache.mirrors.pair.com/tomcat/tomcat-9/v9.0.37/src/
http://apache.mirrors.pair.com/tomcat/tomcat-9/v9.0.37/RELEASE-NOTES
https://tomcat.apache.org/tomcat-9.0-doc/deployer-howto.html
https://tomcat.apache.org/tomcat-9.0-doc/changelog.html
https://tomcat.apache.org/tomcat-9.0-doc/building.html
https://tomcat.apache.org/
http://apache.mirrors.pair.com/tomcat/tomcat-9/v9.0.37/bin/
http://apache.mirrors.pair.com/tomcat/tomcat-9/v9.0.37/?C=S;O=A
http://apache.mirrors.pair.com/tomcat/tomcat-9/v9.0.37/?C=N;O=D
http://apache.mirrors.pair.com/tomcat/tomcat-9/v9.0.37/?C=M;O=A
http://apache.mirrors.pair.com/tomcat/tomcat-9/v9.0.37/?C=D;O=A
http://apache.mirrors.pair.com/tomcat/tomcat-9/v9.0.37/tomcat/tomcat-9/
Parsing http://apache.mirrors.pair.com/tomcat/tomcat-9/v9.0.37/src/
Parsing http://apache.mirrors.pair.com/tomcat/tomcat-9/v9.0.37/RELEASE-NOTES
Parsing https://tomcat.apache.org/tomcat-9.0-doc/deployer-howto.html
Parsing https://tomcat.apache.org/tomcat-9.0-doc/changelog.html
Parsing https://tomcat.apache.org/tomcat-9.0-doc/building.html
Parsing https://tomcat.apache.org/
Parsing http://apache.mirrors.pair.com/tomcat/tomcat-9/v9.0.37/bin/
Parsing http://apache.mirrors.pair.com/tomcat/tomcat-9/v9.0.37/?C=S;O=A
Adding new links to depth 3:
http://apache.mirrors.pair.com/tomcat/tomcat-9/v9.0.37/src/apache-tomcat-9.0.37-src.zip
http://apache.mirrors.pair.com/tomcat/tomcat-9/v9.0.37/src/apache-tomcat-9.0.37-src.tar.gz
http://apache.mirrors.pair.com/tomcat/tomcat-9/v9.0.37/src/?C=S;O=A
http://apache.mirrors.pair.com/tomcat/tomcat-9/v9.0.37/src/?C=N;O=D
http://apache.mirrors.pair.com/tomcat/tomcat-9/v9.0.37/src/?C=M;O=A
http://apache.mirrors.pair.com/tomcat/tomcat-9/v9.0.37/src/?C=D;O=A
http://apache.mirrors.pair.com/tomcat/tomcat-9/v9.0.37/src/tomcat/tomcat-9/v9.0.37/
Bingo!
Minutes elapsed: 0.08
http://apache.mirrors.pair.com/tomcat/tomcat-9/v9.0.37/bin/apache-tomcat-9.0.37.exe
# findDownloadUrl_v0.0.1.ps1
# This little algorithm will parse a web page for downloadable contents, matching given extensions
# Scripty currently is not optimized as multi-threading should greatly improve its performance

$startUrl='http :// apache.mirrors.pair.com/tomcat/tomcat-9/'
$fileExtension='.exe'
$maxDepth=2

function findDownloadUrl{
    param(
        $parentUrl,
        $fileExtension,
        $maxDepth=3
    )
    if(!$parentUrl){
        write-warning "Cannot start with a blank parent URL"
    }elseif($parentUrl -notmatch '/$'){
        $parentUrl=$parentUrl+'/'
        }
    $page=Invoke-WebRequest $parentUrl    
    $links=$page.links.href|Select -Unique|sort -Descending|%{$parentUrl+$_}
    $knownLinks=$links

    function findFile($parentUrl,$extension){
        $ProgressPreference='SilentlyContinue'
        $ErrorActionPreference='stop'
        if($parentUrl -notmatch '/$'){$parentUrl=$parentUrl+'/'}
        #if(!([System.Net.WebRequest]::Create($parentUrl)).GetResponse().StatusCode -eq 200){
        #    return @($false,@()) 
        #}
        try{
            $page=Invoke-WebRequest $parentUrl -TimeoutSec 10
        }catch{
            return @($false,@())
            }
        $newLinks=$page.links.href|?{($_ -notlike "*$(Split-Path $parentUrl -parent)") -and ($_ -notmatch '^http')}| `
            sort -Descending|%{$parentUrl+$(
                                if($_[0] -eq '/'){
                                    $_.Substring(1,$_.length-1)
                                }else{
                                    $_
                                }
                            )}|select -Unique
        $matchedExtension=$newLinks|?{$_ -like "*$extension"}|sort -Descending|select -First 1
        if($matchedExtension){
            return @($true,$matchedExtension)
        }elseif($newLinks){
            return @($false,$newLinks)
        }else{
            return @($false,@())
            } 
    }   
    
    foreach ($link in $links){       
        write-host "Checking $link"                
        $currentDepth=1
        $newLinks=@($link)
        do{            
            $thisLink=$newLinks|Select -Unique|select -First 1
            $newLinks=$newLinks[1..($newLinks.count-1)]
            write-host "Parsing $thisLink"
            $result=findFile $thisLink $fileExtension
            if($result[0]){
                write-host "Bingo!" -ForegroundColor Green
                return $result[1]
            }elseif(($currentDepth++ -le $maxDepth) -and ($result[1]|?{$_ -notin $knownLinks})){                
                $addLinks=$result[1]|?{$_ -notin $knownLinks}
                write-host "Adding new links:`r`n$(($addLinks|out-string).trim())"
                $knownLinks+=$addLinks
                $newLinks=$addLinks+$newLinks
                }
        }until(!$newLinks)
    }

    write-host "$linksChecked links have been checked without any matching file extension $extension" -ForegroundColor Red
    return $false
}
findDownloadUrl $startUrl $fileExtension $maxDepth

PowerShell: Extract Root Domain from URL

# getRootDomainFromUrl.ps1
# This solves the problem of extracting the root domain from a URL
# The challenge here is to incorporate semantics of incorporating the array of TLDs during extraction

$url='https://intranet.kimconnect.co.uk'
$domainsDictionary=@{
    '.ac'='Ascension Island'
    '.ac.uk'='Second-level domain for United Kingdom (.uk) and most often used for academic sites.'
    '.ad'='Andorra'
    '.ae'='United Arab Emirates'
    '.aero'='Air Transportation Industry'
    '.af'='Afghanistan'
    '.ag'='Antigua and Barbuda'
    '.ai'='Anguilla'
    '.al'='Albania'
    '.am'='Armenia'
    '.an'='Netherlands Antilles'
    '.ao'='Angola'
    '.aq'='Antarctica'
    '.ar'='Argentina'
    '.arpa'='Internet infrastructure TLD'
    '.as'='American Somoa'
    '.asia'='Asian countries'
    '.at'='Austria'
    '.au'='Australia'
    '.aw'='Aruba'
    '.ax'='Aland Islands - part of Finland'
    '.az'='Azerbaijan'
    '.ba'='Bosnia and Herzegovinia'
    '.bb'='Barbados'
    '.bd'='Bangladesh'
    '.be'='Belgium'
    '.bf'='Burkina Faso'
    '.bg'='Bulgaria'
    '.bh'='Bahrain'
    '.bi'='Burundi'
    '.biz'='United States business site.'
    '.bj'='Benin'
    '.bm'='Bermuda'
    '.bn'='Brunei Darussalam'
    '.bo'='Bolivia'
    '.br'='Brazil'
    '.bs'='Bahamas'
    '.bt'='Bhutan'
    '.bv'='Bouvet Island'
    '.bw'='Botswana'
    '.by'='Belarus and Byelorussia'
    '.bz'='Belize'
    '.ca'='Canada'
    '.cat'='Catalan'
    '.cc'='Cocos Islands - Keelings'
    '.cd'='Democratic Republic of the Congo'
    '.cf'='Central African Republic'
    '.cg'='Congo'
    '.ch'='Switzerland'
    '.ci'='Cote dIvoire'
    '.ck'='Cook Islands'
    '.cl'='Chile'
    '.cm'='Cameroon'
    '.cn'='China'
    '.co'='Colombia'
    '.co.uk'='Second-level domain for United Kingdom (.uk) and most often used for commercial sites.'
    '.com'='United States commercial website.'
    '.coop'='Business cooperatives and organizations.'
    '.cr'='Costa Rica'
    '.cs'='Former Czechoslovakia'
    '.cu'='Cuba'
    '.cv'='Cape Verde'
    '.cw'='Curaçao'
    '.cx'='Christmas Island'
    '.cy'='Cyprus'
    '.cz'='Czech Republic'
    '.dd'='East Germany'
    '.de'='Germany'
    '.dj'='Djibouti'
    '.dk'='Denmark'
    '.dm'='Dominica'
    '.do'='Dominican Republic'
    '.dz'='Algeria'
    '.ec'='Ecuador'
    '.edu'='United States education site.'
    '.ee'='Estonia'
    '.eg'='Egypt'
    '.eh'='Western Sahara'
    '.er'='Eritrea'
    '.es'='Spain'
    '.et'='Ethiopia'
    '.eu'='European Union'
    '.fi'='Finland'
    '.firm'='Internet site for business or firm.'
    '.fj'='Fiji'
    '.fk'='Falkland Islands and Malvinas'
    '.fm'='Micronesia'
    '.fo'='Faroe Islands'
    '.fr'='France'
    '.fx'='Metropolitan France'
    '.ga'='Gabon'
    '.gb'='Great Britain'
    '.gd'='Grenada'
    '.ge'='Georgia'
    '.gf'='French Guiana'
    '.gg'='Guernsey'
    '.gh'='Ghana'
    '.gi'='Gibraltar'
    '.gl'='Greenland'
    '.gm'='Gambia'
    '.gn'='Guinea'
    '.gov'='United States Government site.'
    '.gov.uk'='Second-level domain for United Kingdom (.uk) and most often used for government sites.'
    '.gp'='Guadeloupe'
    '.gq'='Equatorial Guinea'
    '.gr'='Greece'
    '.gs'='South Georgia and South Sandwich Islands.'
    '.gt'='Guatemala'
    '.gu'='Guam'
    '.gw'='Guinea-Bissau'
    '.gy'='Guyana'
    '.hk'='Hong Kong'
    '.hm'='Heard and McDonald Islands'
    '.hn'='Honduras'
    '.hr'='Croatia/Hrvatska'
    '.ht'='Haiti'
    '.hu'='Hungary'
    '.id'='Indonesia'
    '.ie'='Ireland'
    '.il'='Israel'
    '.im'='Isle of Man'
    '.in'='India'
    '.info'='United States information site with no restrictions.'
    '.int'='International institute site.'
    '.io'='British Indian Ocean Territory'
    '.iq'='Iraq'
    '.ir'='Iran'
    '.is'='Iceland'
    '.it'='Italy'
    '.je'='Jersey - Channel Islands a UK dependency'
    '.jm'='Jamaica'
    '.jo'='Jordan'
    '.jobs'='Job related sites.'
    '.jp'='Japan'
    '.ke'='Kenya'
    '.kg'='Kyrgyzstan'
    '.kh'='Cambodia'
    '.ki'='Kiribati'
    '.km'='Comoros'
    '.kn'='Saint Kitts and Nevis'
    '.kp'='North Korea'
    '.kr'='South Korea'
    '.kw'='Kuwait'
    '.ky'='Cayman Islands'
    '.kz'='Kazakhstan'
    '.la'='Laos'
    '.lb'='Lebanon'
    '.lc'='Saint Lucia'
    '.li'='Liechtenstein'
    '.lk'='Sri Lanka'
    '.lr'='Liberia'
    '.ls'='Lesotho'
    '.lt'='Lithuania'
    '.ltd.uk'='Second-level domain for United Kingdom (.uk) and most often used for limited company sites.'
    '.lu'='Luxembourg'
    '.lv'='Latvia'
    '.ly'='Libya'
    '.ma'='Morocco'
    '.mc'='Monaco'
    '.md'='Moldova'
    '.me'='Montenegro'
    '.me.uk'='Second-level domain for United Kingdom (.uk) and most often used for personal sites.'
    '.mg'='Madagascar'
    '.mh'='Marshall Islands'
    '.mil'='United States Military site.'
    '.mk'='Macedonia'
    '.ml'='Mali'
    '.mm'='Myanmar'
    '.mn'='Mongolia'
    '.mo'='Macau'
    '.mobi'='Mobile devices'
    '.mod.uk'='Second-level domain for United Kingdom (.uk) and most often used for military of defence sites.'
    '.mp'='Northern Mariana Islands'
    '.mq'='Martinique'
    '.mr'='Mauritania'
    '.ms'='Montserrat'
    '.mt'='Malta'
    '.mu'='Mauritius'
    '.museum'='Worldwide museums'
    '.mv'='Maldives'
    '.mw'='Malawi'
    '.mx'='Mexico'
    '.my'='Malaysia'
    '.mz'='Mozambique'
    '.na'='Namibia'
    '.name'='Individual and family names'
    '.nato'='NATO site.'
    '.nc'='New Caledonia'
    '.ne'='Niger'
    '.net'='United States Internet administrative site. See the .net definition for alternative definitions.'
    '.net.uk'='Second-level domain for United Kingdom (.uk) and most often used for network company sites.'
    '.nf'='Norfolk Island'
    '.ng'='Nigeria'
    '.nhs.uk'='Second-level domain for United Kingdom (.uk) and most often used for national health service institutions'
    '.ni'='Nicaragua'
    '.nl'='Netherlands'
    '.no'='Norway'
    '.nom'='Personal site'
    '.np'='Nepal'
    '.nr'='Nauru'
    '.nt'='Neutral Zone'
    '.nu'='Niue'
    '.nz'='New Zealand'
    '.om'='Oman'
    '.org'='Organization (non-profit) sites.'
    '.org.uk'='Second-level domain for United Kingdom (.uk) and most often used for non-profit sites.'
    '.pa'='Panama'
    '.pe'='Peru'
    '.pf'='French Polynesia'
    '.pg'='Papua New Guinea'
    '.ph'='Philippines'
    '.pk'='Pakistan'
    '.pl'='Poland'
    '.plc.uk'='Second-level domain for United Kingdom (.uk) and most often used for public limited company sites.'
    '.pm'='St. Pierre and Miquelon'
    '.pn'='Pitcairn'
    '.post'='sTLD (sponsored top-level domain) available exclusively for the postal sector.'
    '.pr'='Puerto Rico'
    '.pro'='United States professional site for accountants'
    '.ps'='Palestinian territories'
    '.pt'='Portugal'
    '.pw'='Palau'
    '.py'='Paraguay'
    '.qa'='Qatar'
    '.re'='Reunion'
    '.ro'='Romania'
    '.rs'='Republic of Serbia'
    '.ru'='Russian Federation'
    '.rw'='Rwanda'
    '.sa'='Saudi Arabia'
    '.sb'='Solomon Islands'
    '.sc'='Seychelles'
    '.sch.uk'='Second-level domain for United Kingdom (.uk) and most often used for school sites.'
    '.sd'='Sudan'
    '.se'='Sweden'
    '.sg'='Singapore'
    '.sh'='Saint Helena'
    '.si'='Slovenia'
    '.sj'='Svalbard and Jan Mayen Islands'
    '.sk'='Slovakia'
    '.sl'='Sierra Leone'
    '.sm'='San Marino'
    '.sn'='Senegal'
    '.so'='Somalia'
    '.sr'='Suriname'
    '.ss'='South Sudan'
    '.st'='Sao Tome and Principe'
    '.store'='United States domain for retail business site.'
    '.su'='Former USSR'
    '.sv'='El Salvador'
    '.sy'='Syria'
    '.sz'='Swaziland'
    '.tc'='Turks and Caicos Islands'
    '.td'='Chad'
    '.tel'='Internet communication services'
    '.tf'='French Southern Territory and Antarctic Lands.'
    '.tg'='Togo'
    '.th'='Thailand'
    '.tj'='Tajikistan'
    '.tk'='Tokelau'
    '.tl'='East Timor'
    '.tm'='Turkmenistan'
    '.tn'='Tunisia'
    '.to'='Tonga'
    '.tp'='East Timor'
    '.tr'='Turkey'
    '.travel'='Travel related sites.'
    '.tt'='Trinidad and Tobago'
    '.tv'='Tuvalu'
    '.tw'='Taiwan'
    '.tz'='Tanzania'
    '.ua'='Ukraine'
    '.ug'='Uganda'
    '.uk'='United Kingdom'
    '.um'='United States minor outlying islands.'
    '.us'='United States'
    '.uy'='Uruguay'
    '.uz'='Uzbekistan'
    '.va'='Vatican City State'
    '.vc'='Saint Vincent and the Grenadines'
    '.ve'='Venezuela'
    '.vg'='British Virgin Islands'
    '.vi'='United States Virgin Islands'
    '.vn'='Vietnam'
    '.vu'='Vanuatu'
    '.web'='Internet site about the World Wide Web.'
    '.wf'='Wallis and Futuna Islands'
    '.ws'='Samoa'
    '.xxx'='Adult entertainment domain'
    '.ye'='Yemen'
    '.yt'='Mayotte'
    '.yu'='Yugoslavia'
    '.za'='South Africa'
    '.zm'='Zambia'
    '.zr'='Zaire'
    '.zw'='Zimbabwe'
}

function getRootDomain($url){
    $domain=([uri]$url).Host
    $matchedTwoDottedDomain=$domainsDictionary.keys|?{$domain -match "$_$"}|?{$_ -match '\.\w+\.'}
    $rootDomain=if(!$matchedTwoDottedDomain){$domain.split('.')[-2..-1] -join '.'}
                else{$domain.split('.')[-3..-1] -join '.'}
    return $rootDomain
}

getRootDomain $url

PowerShell: Use Selenium Drivers to Automate Logins

Set Variables:

$username='testrobot'
$password='password'
$url='https://kimconnect.com'

# Version 0.0.2 - return true or false, while keeping IE Open for further use
# This iteration also handles IE Protected mode errors

# Usage:
# 1. $ie=autologinSe "$url" "$username" "$password"
# 2. autologinSe $someUrlNoLogin
# 3. autologinSe "$url" "$username" "$password" -exitIeWhenDone $true

Run the Function:

function autologinSe{
    param(
        $url,
        $username,
        $password,
        $usernameElementId='userNameInput',
        $passwordElementId='passwordInput',
        $submitButtonId='submitButton',
        $exitIeWhenDone=$false
    )
    $ErrorActionPreference = 'continue'

    # Initial validation
    if(!$url){write-warning "No URL specified.";return $false}

    function killInternetExplorer{
        $ieInstances=(New-Object -COM 'Shell.Application').Windows()|?{$_.Name -like '*Internet Explorer*'} 
        $ieInstances|%{$_.Quit()
        [Runtime.Interopservices.Marshal]::ReleaseComObject($_)
        }
        [GC]::Collect()
        [GC]::WaitForPendingFinalizers()
        }

    function enableIeProtectedMode{
        # $hives = 0..4|%{"HKLM:\Software\Microsoft\Windows\CurrentVersion\Internet Settings\Zones\$_"}
        $hives = 0..4|%{"HKCU:\Software\Microsoft\Windows\CurrentVersion\Internet Settings\Zones\$_"}
        $keyName='2500' # Key Name '2500' corresponds to 'Protected Mode' in IE
        
        #Skipping zone 0 as that is the default local machine zone
        $hives[1..4]|%{Set-ItemProperty -Path $_ -Name $keyName -Value 0}
        $keys=$hives|%{Get-ItemProperty -Path $_}|select DisplayName, `
                                                        @{name='status';e={
                                                                        if($_.$keyName -eq 0){'enabled'}
                                                                        elseif($_.$keyName -eq 3){'disabled'}
                                                                        else{'n/a'}                                                                                        
                                                                        }}
        write-host "IE Protected Mode Standardized Values:`r`n$($keys|out-string)" 
    }

    function disableIeProtectedMode{
        # $hives = 0..4|%{"HKLM:\Software\Microsoft\Windows\CurrentVersion\Internet Settings\Zones\$_"}
        $hives = 0..4|%{"HKCU:\Software\Microsoft\Windows\CurrentVersion\Internet Settings\Zones\$_"}
        $keyName='2500' # Key Name '2500' corresponds to 'Protected Mode' in IE
        
        #Skipping zone 0 as that is the default local machine zone
        $hives[1..4]|%{Set-ItemProperty -Path $_ -Name $keyName -Value 3}
        $keys=$hives|%{Get-ItemProperty -Path $_}|select DisplayName, `
                                                        @{name='status';e={
                                                                        if($_.$keyName -eq 0){'enabled'}
                                                                        elseif($_.$keyName -eq 3){'disabled'}
                                                                        else{'n/a'}                                                                                        
                                                                        }}
        write-host "IE Protected Mode Standardized Values:`r`n$($keys|out-string)" 
    }

    function allowActiveX($zone='Trusted'){
        #Source: http://support.microsoft.com/KB/182569
        $zoneCode=switch($zone){
            'My Computer'{0;break}
            'Local Intranet'{1;break}
            'Trusted'{2;break}
            'Internet'{3;break}
            'Restricted Sites'{4;break}
            default{2}
            }
        #Reference table:
        #Value    Setting
        #------------------------------
        #0        My Computer
        #1        Local Intranet Zone
        #2        Trusted sites Zone
        #3        Internet Zone
        #4        Restricted Sites Zone
        $hashMap=@{
            '2702'=0 #ActiveX controls and plug-ins: Allow ActiveX Filtering = Enable (2702)
            '1208'=0 #ActiveX controls and plug-ins: Allow previously unused ActiveX controls to run without prompt = Enable (1208)
            '1209'=0 #ActiveX controls and plug-ins: Allow Scriptlets = Enable (1209)
            '2201'=3 #ActiveX controls and plug-ins: Automatic prompting for ActiveX controls = Disable (2201)
            '2000'=0 #ActiveX controls and plug-ins: Binary and script behaviors = Enable (2000)
            '120A'=0 #Display video and animation on a webpage that does not use external media player = Enable (120A)
            '1001'=0 #ActiveX controls and plug-ins: Download signed ActiveX controls = Enable (1001)
            '1004'=0 #ActiveX controls and plug-ins: Download unsigned ActiveX controls = Enable (1004)
            '1201'=0 #ActiveX controls and plug-ins: Initialize and script ActiveX controls not marked as safe for scripting = Enable (1201)
            '120B'=3 #Only allow approved domains to use ActiveX without prompt = Disable (120B)
            '1200'=0 #ActiveX controls and plug-ins: Run ActiveX controls and plug-ins = Enable (1200)
            '1405'=0 #ActiveX controls and plug-ins: Script ActiveX controls marked as safe for scripting = Enable (1405)
            }        
        
        $trustedDomains="HKCU:\Software\Microsoft\Windows\CurrentVersion\Internet Settings\zones\$zoneCode"
        $currentValues=Get-ItemProperty $trustedDomains
        foreach ($item in $hashMap.GetEnumerator()) {
            $key = $item.Key
            $value = $item.Value
            if($currentValues.$key -ne $value){
                    New-ItemProperty -Path $trustedDomains -Name $key -Value $value -PropertyType DWORD -Force
                }
        }
    }

    function addDomainToTrustedSites($url){
        $httpType=.{[void]($url -match '^(https{0,1})');$matches[1]}
        $domain=([uri]$url).Host
        #$rootDomain=$domain.split('.')[-2..-1] -join '.' # This is assuming that the TLD is one-dotted (e.g. .com) not two-dotted (e.g. co.uk)
        $rootDomain=.{$fragments=$domain.split('.')
                    $fragments[1..$($fragments.count)] -join '.'
                    }
        write-host "Root domain detected`t: $rootDomain"        
        # The more advanced function to retrieve this value is at https://kimconnect.com/powershell-extract-root-domain-from-url
        if ($rootDomain -notmatch '\.' -or $rootDomain -eq $env:USERDNSDOMAIN){
            write-host "There's no need to add $url to the Trusted zone as it is local to this domain."
            return $true
            }
        $dwordValue=2 # value of true correlates to 'enable'
        $domainRegistryPath='HKCU:\Software\Microsoft\Windows\CurrentVersion\Internet Settings\ZoneMap\Domains'
        $domainRegistryPath2='HKCU:\Software\Microsoft\Windows\CurrentVersion\Internet Settings\ZoneMap\EscDomains' #EscDomains key applies to those protocols that are affected by the Enhanced Security Configuration (ESC)
        $null=New-Item -Path 'HKCU:\Software\Microsoft\Windows\CurrentVersion\Internet Settings\ZoneMap' -ItemType File -Name 'EscDomains' -Force
        $null=New-Item -Path "$domainRegistryPath" -ItemType File -Name "$rootDomain" -Force
        $null=New-Item -Path "$domainRegistryPath2" -ItemType File -Name "$rootDomain" -Force
        $null=Set-ItemProperty -Path "$domainRegistryPath\$rootDomain" -Name $httpType -Value $dwordValue
        $null=Set-ItemProperty -Path "$domainRegistryPath2\$rootDomain" -Name $httpType -Value $dwordValue

        # Also add {about:blank} record as that doesn't seem to have been added by default
        if (!(test-path "$domainRegistryPath\blank")){
            #New-ItemProperty -Path $trustedDomains -Name $key -Value $value -PropertyType DWORD -Force
            $null=New-Item -Path "$domainRegistryPath" -ItemType File -Name 'blank'
            $null=Set-ItemProperty -Path "$domainRegistryPath\blank" -Name 'about' -Value $dwordValue
            }
        if (!(test-path "$domainRegistryPath2\blank")){
            $null=New-Item -Path "$domainRegistryPath2" -ItemType File -Name 'blank'
            $null=Set-ItemProperty -Path "$domainRegistryPath2\blank" -Name 'about' -Value $dwordValue
            }                     

        # Also add {about:internet} record since it will stop a login when missing
        if (!(test-path "$domainRegistryPath\internet")){
            $null=New-Item -Path "$domainRegistryPath" -ItemType File -Name 'internet'
            $null=Set-ItemProperty -Path "$domainRegistryPath\internet" -Name 'about' -Value $dwordValue
            }
        if (!(test-path "$domainRegistryPath2\internet")){
            $null=New-Item -Path "$domainRegistryPath2" -ItemType File -Name 'internet'
            $null=Set-ItemProperty -Path "$domainRegistryPath2\internet" -Name 'about' -Value $dwordValue
            } 
            
        $valueAfterChanged=(Get-ItemProperty "$domainRegistryPath\$rootDomain")."$httpType"
        $value2AfterChanged=(Get-ItemProperty "$domainRegistryPath2\$rootDomain")."$httpType"
        if ($valueAfterChanged -eq 2 -and $value2AfterChanged -eq 2 ){
            write-host "$rootDomain has been added to Internet Explorer"
            return $true
            }
        else{
            write-warning "$rootDomain has NOT been added to Internet Explorer."
            return $false
            }
    }

    function includeSelenium{
        Import-Module Selenium -ea SilentlyContinue
        if (!(get-module selenium -EA SilentlyContinue)){
            Start-job {
                [Net.ServicePointManager]::SecurityProtocol = [Net.SecurityProtocolType]::Tls12
                if(!(Get-PackageProvider Nuget -ea SilentlyContinue)){Install-PackageProvider -Name NuGet -Force}
                # Defining $ENV:ChocotaleyInstall so that it would be called by refreshenv
                $ENV:ChocolateyInstall = Convert-Path "$((Get-Command choco).Path)\..\.."   
                Import-Module "$env:ChocolateyInstall\helpers\chocolateyProfile.psm1"
                Install-Module Selenium -Force -Confirm:$False
                } |Receive-Job -Wait
            Update-SessionEnvironment
            Import-Module Selenium
            }
        }

    function invokeSelenium($url,$userName,$password,$usernameElementId,$passwordElementId,$submitButtonId){
        $ErrorActionPreference = "Stop"
        function closeSelenium($selenium){
            if($selenium){
                $selenium.close()
                $selenium.quit()
                }
            }
        
        function noLogin($url){
            $seleniumIe = New-Object "OpenQA.Selenium.IE.InternetExplorerDriver"
            $seleniumIe.Navigate().GoToURL($url)
            $title=$seleniumIe.Title
            write-host "Page reached: '$title'"
            $trustedSiteError=$title -match '^Error'
            if($trustedSiteError){
                write-host "An site trust issue has been detected. Adding root domain to the trusted sites list to resolve this issue."
                addDomainToTrustedSites $url              
                closeSelenium $seleniumIe                
                return $false
                }
            else{
                return $seleniumIe                
                }
            }

        function login($url,$username,$password,$usernameElementId,$passwordElementId,$submitButtonId){
            $seleniumIe = New-Object "OpenQA.Selenium.IE.InternetExplorerDriver"
            $seleniumIe.Navigate().GoToURL($url)
            $userField=$seleniumIe.FindElementById($usernameElementId)
            $userField.clear()
            $userField.SendKeys($username)
            $passwordField=$seleniumIe.FindElementById($passwordElementId)
            $passwordField.SendKeys('')
            $passwordField.clear()           
            $passwordField.SendKeys($password)
            $submitButton=$seleniumIe.FindElementById($submitButtonId)
            $submitButton.Click()
            $title=$seleniumIe.Title
            write-host "Page reached: '$title'"
            $trustedSiteError=$title -match '^Error'
            if($trustedSiteError){
                write-warning "A site trust issue has been detected."                
                closeSelenium $seleniumIe                
                return $false
            }else{
                return $seleniumIe                
                }
            }
        
        try{
            $null=allowActiveX
            $isLogin=$userName,$password,$usernameElementId,$passwordElementId,$submitButtonId|?{!(!$_)}
            if($isLogin){
                write-host "Login to $url as $userName..."
                $ie=login $url $userName $password $usernameElementId $passwordElementId $submitButtonId
            }else{
                write-host "Accesing $url without login..."
                $ie=nologin $url
                }
            return $ie
            }
        catch{            
            Write-Warning $Error[0].Exception.Message
            return $false
            }
        }

    try{
        write-host "Username`t: $username`r`nPassword`t: $(!(!$password))`r`nusernameElementId`t: $usernameElementId`r`npasswordElementId`t: $passwordElementId`r`nsubmitButtonId`t: $submitButtonId"
        $null=includeSelenium
        $null=disableIeProtectedMode
        $null=addDomainToTrustedSites $url                   
        if(get-module selenium -ea SilentlyContinue){
            $isLogin=$userName,$password,$usernameElementId,$passwordElementId,$submitButtonId|?{!(!$_)}
            if($isLogin){                
                $selenium=invokeSelenium $url $userName $password $usernameElementId $passwordElementId $submitButtonId
            }else{
                write-host "No username or password are given. Proceeding to access only the provided URL."
                $selenium=invokeSelenium $url
                }
        }else{
            write-warning "Please manually verify that the Selenium module is installed before retrying this function."
            }
        if($selenium){            
            if($exitIeWhenDone){
                $null=killInternetExplorer
                return $true
            }else{
                return $selenium
                }
        }else{
            write-warning "There were errors preventing a successful login."
            return $false
            }
        }
    catch {
        write-warning "$_"        
        return $false
        }
    
    # Note on a common error:
    #New-Object : Exception calling ".ctor" with "0" argument(s): "Unexpected error launching Internet Explorer. Protected
    #Mode settings are not the same for all zones. Enable Protected Mode must be set to the same value (enabled or
    #disabled) for all zones. (SessionNotCreated)"
    #At line:1 char:15
    #+ $seleniumIe = New-Object "OpenQA.Selenium.IE.InternetExplorerDriver"
    #+               ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    #    + CategoryInfo          : InvalidOperation: (:) [New-Object], MethodInvocationException
    #    + FullyQualifiedErrorId : ConstructorInvokedThrowException,Microsoft.PowerShell.Commands.NewObjectCommand
    # Solution: either DISABLE or ENABLE Protected mode for ALL ZONES
}

$ie=autoLoginSe $url $username $password

Deprecated version:

# Version 0.0.1 - function to return true or false, and close IE afterward
function autologinSe{
    param(
        $url,
        $username,
        $password,
        $usernameElementId='userNameInput',
        $passwordElementId='passwordInput',
        $submitButtonId='submitButton'
    )
    $ErrorActionPreference = "Stop"

    function killInternetExplorer{
        $ieInstances=(New-Object -COM 'Shell.Application').Windows()|?{$_.Name -like '*Internet Explorer*'} 
        $ieInstances|%{$_.Quit()
        [Runtime.Interopservices.Marshal]::ReleaseComObject($_)
        }
        [GC]::Collect()
        [GC]::WaitForPendingFinalizers()
        }

    try{
        killInternetExplorer|out-null
        if (!(get-module selenium -EA SilentlyContinue)){
            [Net.ServicePointManager]::SecurityProtocol = [Net.SecurityProtocolType]::Tls12
            Install-Module Selenium -Force
            }
        Import-Module Selenium
        $seleniumIe = New-Object "OpenQA.Selenium.IE.InternetExplorerDriver"
        $seleniumIe.Navigate().GoToURL($url)
        $userField=$seleniumIe.FindElementById($usernameElementId)
        $userField.SendKeys($username)
        $passwordField=$seleniumIe.FindElementById($passwordElementId)
        $passwordField.SendKeys($password)
        $submitButton=$seleniumIe.FindElementById($submitButtonId)
        $submitButton.Click()
        $title=$seleniumIe.Title
        write-host "Page reached: '$title'"
        $seleniumIe.close()
        $seleniumIe.quit()
        $not404=$title -notmatch '^404|^Error'
        if($not404){return $true}
        else{return $false}
        }
    catch{
        write-warning "$($error[0])"
        return $false
        }
    # Troubleshooting:
    #New-Object : Exception calling ".ctor" with "0" argument(s): "Unexpected error launching Internet Explorer. Protected
    #Mode settings are not the same for all zones. Enable Protected Mode must be set to the same value (enabled or
    #disabled) for all zones. (SessionNotCreated)"
    #At line:1 char:15
    #+ $seleniumIe = New-Object "OpenQA.Selenium.IE.InternetExplorerDriver"
    #+               ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    #    + CategoryInfo          : InvalidOperation: (:) [New-Object], MethodInvocationException
    #    + FullyQualifiedErrorId : ConstructorInvokedThrowException,Microsoft.PowerShell.Commands.NewObjectCommand
    # Solution: either DISABLE or ENABLE Protected mode for ALL ZONES
}
autoLoginSe $url $username $password
# Troubleshooting:
#New-Object : Exception calling ".ctor" with "0" argument(s): "Unexpected error launching Internet Explorer. Protected
#Mode settings are not the same for all zones. Enable Protected Mode must be set to the same value (enabled or
#disabled) for all zones. (SessionNotCreated)"
#At line:1 char:15
#+ $seleniumIe = New-Object "OpenQA.Selenium.IE.InternetExplorerDriver"
#+               ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
#    + CategoryInfo          : InvalidOperation: (:) [New-Object], MethodInvocationException
#    + FullyQualifiedErrorId : ConstructorInvokedThrowException,Microsoft.PowerShell.Commands.NewObjectCommand
# Solution: either DISABLE or ENABLE Protected mode for ALL ZONES

How to Remove Question Mark ‘Info’ Button in SyntaxHighlighter

If you’re looking for a way to remove this subtle, yet spammy link:

Do this:
Goto Dashboard > Appearance > Customize > Additional CSS

Simply paste this code:

.syntaxhighlighter .toolbar {
    /*color: white !important;
    background: #6ce26c !important;
    border: none !important;*/
    display: none;
}

Yo welcomed, homie.

Updating SSL Certificates on Active Directory Federation Services (ADFS) Server

Scripted Version Notes:

Part 1: Adding Cert to ADFS Server

# Import Cert and get its thumbprint
$certPath="C:\wildcard_kimconnectcom.pfx"
$certPlaintextPassword='PASSWORDHERE'
$certEncryptedPassword=ConvertTo-SecureString $certPlaintextPassword -AsPlainText -Force
$importedCert=Import-PfxCertificate -CertStoreLocation Cert:\LocalMachine\My -FilePath $certPath -Password $certEncryptedPassword
$importedCertThumbprint=$importedCert.Thumbprint

# Get service account running adfs
$adfsService=Get-Wmiobject win32_service|where-object{$_.name -eq 'adfssrv'}
$adfsRunasUser=$adfsService.StartName

# Grant permissions
$certLocation='Cert:\LocalMachine\My'
$permissions='full'
$access='allow'
$rule = new-object security.accesscontrol.filesystemaccessrule $adfsRunasUser,$permissions,$access
$root = "c:\programdata\microsoft\crypto\rsa\machinekeys"
$thisCert = ls $certLocation|? {$_.thumbprint -eq $importedCertThumbprint}
$keyname = $thisCert.privatekey.cspkeycontainerinfo.uniquekeycontainername
$keyPath = [io.path]::combine($root, $keyname)
if ([io.file]::exists($keyPath)){
	$acl = get-acl -path $keyPath
	$acl.addaccessrule($rule)
	write-host $keyPath
	set-acl $keyPath $acl
}else{
	write-warning "$keyPath NOT Found!"
}

# Add Public Cert to ADFS Server
# Note: changes made in the GUI would not change the configuration based on the HTTP.sys. Hence, the configuration cmdlets below are necessary
# Set-ADFSCertificate -Thumbprint $importedCertThumbprint # This command deals with local certs
Set-AdfsSslCertificate -Thumbprint $importedCertThumbprint
Set-AdfsCertificate -CertificateType 'Service-Communications' -Thumbprint $importedCertThumbprint
#Set-ADFSCertificate -IsPrimary -CertificateType 'Token-Decrypting' -Thumbprint $importedCertThumbprint
#Set-ADFSCertificate -IsPrimary -CertificateType 'Token-Signing' -Thumbprint $importedCertThumbprint
# Set-ADFSCertificate : PS0010: You must add the certificate before you can set it to be the primary certificate.
# At line:1 char:1
# + Set-ADFSCertificate -IsPrimary -CertificateType 'Token-Decrypting' -Thumbprint $ ...
# + ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
#     + CategoryInfo          : InvalidArgument: (:) [Set-AdfsCertificate], ArgumentException
#     + FullyQualifiedErrorId : PS0010,Microsoft.IdentityServer.Management.Commands.SetCertificateCommand
Set-ADFSProperties -AutoCertificateRollover $true
Restart-Service ADFSSRV # restart-service $adfsService.Name
# Update-ADFSCertificate -CertificateType token-signing # Optional

# Validation
Get-AdfsSslCertificate # Check SSL certs
Get-ADFSCertificate -CertificateType token-signing # Check token signing cert

Part 2: Adding Cert to Web Application Server

# Import Cert to WAP Server
$certPath="C:\wildcard_kimconnectcom.pfx"
$certPlaintextPassword='PASSWORDHERE'
$certEncryptedPassword=ConvertTo-SecureString $certPlaintextPassword -AsPlainText -Force
$importedCert=Import-PfxCertificate -CertStoreLocation Cert:\LocalMachine\My -FilePath $certPath -Password $certEncryptedPassword
$importedCertThumbprint=$importedCert.Thumbprint
Set-WebApplicationProxySslCertificate -Thumbprint $importedCertThumbprint

# Non-public Cert deployment - only applicable
# Install-WebApplicationProxy -CertificateThumbprint $importedCertThumbprint -FederationServiceName sts.kimconnect.com

# Validate
$externalCertThumbprint=(Get-WebApplicationProxyApplication).ExternalCertificateThumbprint
if($externalCertThumbprint -eq $importedCertThumbprint){
    write-host 'Success'
}else{
    write-host "External Cert thumbprint $externalCertThumbprint doesn't match $importedCertThumbprint"
}
Legacy Notes…

1. Adding Tool to enable AD-FS claims visbility:

# Add ClaimsXray (https://adfshelp.microsoft.com/ClaimsXray/TokenRequest)
$authzRules = "=>issue(Type = `"http://schemas.microsoft.com/authorization/claims/permit`", Value = `"true`"); "
$issuanceRules = "@RuleName = `"Issue all claims`"`nx:[]=>issue(claim = x); "
$redirectUrl = "https://adfshelp.microsoft.com/ClaimsXray/TokenResponse"
$samlEndpoint = New-AdfsSamlEndpoint -Binding POST -Protocol SAMLAssertionConsumer -Uri $redirectUrl
Add-ADFSRelyingPartyTrust -Name "ClaimsXray" -Identifier "urn:microsoft:adfs:claimsxray" -IssuanceAuthorizationRules $authzRules -IssuanceTransformRules $issuanceRules -WSFedEndpoint $redirectUrl -SamlEndpoint $samlEndpoint

2. Check IDP: use a browser to navigate to https://sts.kimconnect.com/adfs/ls/idpinitiatedsignon.aspx

3. Updating Self Signed Certs:

# Check Self Signed Certificates of ADFS. If AutoCertificateRollover = True, do nothing as certs will auto roll.
PS C:\Windows\system32> get-adfsproperties
AcceptableIdentifiers : {}
AddProxyAuthorizationRules : exists([Type ==
"http://schemas.microsoft.com/ws/2008/06/identity/claims/groupsid", Value
== "S-1-5-32-544", Issuer =~ "^AD AUTHORITY$"]) => issue(Type =
"http://schemas.microsoft.com/authorization/claims/permit", Value =
"true");
c:[Type ==
"http://schemas.microsoft.com/ws/2008/06/identity/claims/primarysid",
Issuer =~ "^AD AUTHORITY$" ]
=> issue(store="_ProxyCredentialStore
",types=("http://schemas.micr
osoft.com/authorization/claims/permit"),query="isProxyTrustManagerSid({0})
", param=c.Value );
c:[Type ==
"http://schemas.microsoft.com/ws/2008/06/identity/claims/proxytrustid",
Issuer =~ "^SELF AUTHORITY$" ]
=> issue(store="_ProxyCredentialStore
",types=("http://schemas.micr
osoft.com/authorization/claims/permit"),query="isProxyTrustProvisioned({0}
)", param=c.Value );
ArtifactDbConnection : Data Source=tor-sql-node02.corp.kimconnect.com;Initial
Catalog=AdfsArtifactStore;Integrated Security=True;Min Pool Size=20
AuthenticationContextOrder : {urn:oasis:names:tc:SAML:2.0:ac:classes:Password,
urn:oasis:names:tc:SAML:2.0:ac:classes:PasswordProtectedTransport,
urn:oasis:names:tc:SAML:2.0:ac:classes:TLSClient,
urn:oasis:names:tc:SAML:2.0:ac:classes:X509...}
AutoCertificateRollover : True
CertificateCriticalThreshold : 2
CertificateDuration : 365
CertificateGenerationThreshold : 20
CertificatePromotionThreshold : 5
CertificateRolloverInterval : 720
CertificateSharingContainer : CN=af389bbd-b0c0-405c-b965-bca5e7aa02e5,CN=ADFS,CN=Microsoft,CN=Program
Data,DC=corp,DC=kimconnect,DC=com
CertificateThresholdMultiplier : 1440
ClientCertRevocationCheck : None
ContactPerson : Microsoft.IdentityServer.Management.Resources.ContactPerson
DisplayName : kimconnect
IntranetUseLocalClaimsProvider : False
ExtendedProtectionTokenCheck : Allow
FederationPassiveAddress : /adfs/ls/
HostName : sts.kimconnect.com
HttpPort : 80
HttpsPort : 443
TlsClientPort : 49443
Identifier : http://sts.kimconnect.com/adfs/services/trust
InstalledLanguage : en-US
LogLevel : {Errors, Information, Verbose, Warnings}
MonitoringInterval : 1440
NetTcpPort : 1501
NtlmOnlySupportedClientAtProxy : True
OrganizationInfo :
PreventTokenReplays : True
ProxyTrustTokenLifetime : 21600
ReplayCacheExpirationInterval : 60
SignedSamlRequestsRequired : False
SamlMessageDeliveryWindow : 5
SignSamlAuthnRequests : False
SsoLifetime : 480
PersistentSsoLifetimeMins : 10080
KmsiLifetimeMins : 1440
PersistentSsoEnabled : True
PersistentSsoCutoffTime : 1/1/0001 12:00:00 AM
KmsiEnabled : False
LoopDetectionEnabled : True
LoopDetectionTimeIntervalInSeconds : 20
LoopDetectionMaximumTokensIssuedInInterval : 5
PasswordValidationDelayInMinutes : 60
SendClientRequestIdAsQueryStringParameter : False
WIASupportedUserAgents : {MSAuthHost/1.0/In-Domain, MSIE 6.0, MSIE 7.0, MSIE 8.0...}
ExtranetLockoutThreshold : 2147483647
ExtranetLockoutEnabled : False
ExtranetObservationWindow : 00:30:00
GlobalRelyingPartyClaimsIssuancePolicy : c:[Type == "http://schemas.microsoft.com/2012/01/devicecontext/claims/isre
gistereduser"] => issue(claim = c);c:[Type ==
"http://schemas.microsoft.com/2012/01/devicecontext/claims/identifier"]
=> issue(claim = c);
PromptLoginFederation : FallbackToProtocolSpecificParameters
PromptLoginFallbackAuthenticationType : urn:oasis:names:tc:SAML:1.0:am:password

4. Updating Public Certs:
– Add new public Cert into Local Machine
– Check Public Certificates of Local Machine

PS C:\Windows\system32> dir cert:LocalMachine\My
PSParentPath: Microsoft.PowerShell.Security\Certificate::LocalMachine\My
Thumbprint Subject
---------- -------
1234567890ABCDEFGHIJKLMNOPQRSTUVWXYZ CN=DigiCert SHA2 High Assurance Server CA, OU=www.digicert.com, O=DigiCert...
1234567890ABCDEFGHIJKLMNOPQRSTUVWXYZ CN=ADFS01, CN=2a362dd8-a989-4573-b06c-b9515bfe5448, OU=Microsoft ADFS ...
1234567890ABCDEFGHIJKLMNOPQRSTUVWXYZ CN=ADFS01, CN=2a362dd8-a989-4573-b06c-b9515bfe5448, OU=Microsoft ADFS ...
1234567890ABCDEFGHIJKLMNOPQRSTUVWXYZ CN=*.kimconnect.com, O="Kim Connect, Inc.", L=Torrance, S=CA, C=US
1234567890ABCDEFGHIJKLMNOPQRSTUVWXYZ CN=ADFS Encryption - sts.kimconnect.com
1234567890ABCDEFGHIJKLMNOPQRSTUVWXYZ CN=DigiCert High Assurance EV Root CA, OU=www.digicert.com, O=DigiCert Inc...
1234567890ABCDEFGHIJKLMNOPQRSTUVWXYZ CN=ADFS Signing - sts.kimconnect.com
1234567890ABCDEFGHIJKLMNOPQRSTUVWXYZ CN=ADFS01, CN=2a362dd8-a989-4573-b06c-b9515bfe5448, OU=Microsoft ADFS ...
# Check Certificate association on ADFS

PS C:\Windows\system32> get-adfssslcertificate

HostName PortNumber CertificateHash
-------- ---------- ---------------
localhost 443 1234567890ABCDEFGHIJKLMNOPQRSTUVWXYZ
sts.kimconnect.com 443 1234567890ABCDEFGHIJKLMNOPQRSTUVWXYZ
sts.kimconnect.com 49443 1234567890ABCDEFGHIJKLMNOPQRSTUVWXYZ
localhost 444 1234567890ABCDEFGHIJKLMNOPQRSTUVWXYZ
sts.kimconnect.com 444 1234567890ABCDEFGHIJKLMNOPQRSTUVWXYZ
# Update Pulblic Certificates in ADFS
set-adfssslcertificate -thumbprint <your newcert thumbprint>

5. Validate Relying Party Trusts
– CRM Claims Relying Party:
— Purpose: CRM main login authentication
— Pull federation metadata from https://auth.kimconnect.com/FederationMetadata/2007-06/FederationMetadata.xml
— Claim Rules: Pass through UPN, Pass Through Primary SID, Transform Windows Account Name
– CRM IFD Relying Party:
— Purpose: CRM sub-modules authentication
— Pull federation metadata from https://crm.kimconnect.com/FederationMetadata/2007-06/FederationMetadata.xml
— Claim Rules: Pass through UPN, Pass Through Primary SID, Transform Windows Account Name

PowerShell: Convert Between Various SSL Certificate Formats

# Install Choco (look for instructions in this blog)

# Install openssl.light
choco install openssl.light -y
# Defining $ENV:ChocotaleyInstall so that it would be called by refreshenv
$ENV:ChocolateyInstall = Convert-Path "$((Get-Command choco).Path)\..\.."
Import-Module "$env:ChocolateyInstall\helpers\chocolateyProfile.psm1"
Update-SessionEnvironment
# Convert PEM to DER
openssl x509 -outform der -in certificate.pem -out certificate.der

# Convert PEM to P7B
openssl crl2pkcs7 -nocrl -certfile certificate.cer -out certificate.p7b -certfile CACert.cer

# Convert PEM to PFX
openssl pkcs12 -export -out certificate.pfx -inkey privateKey.key -in certificate.crt -certfile CACert.crt

# Convert DER to PEM
openssl x509 -inform der -in certificate.cer -out certificate.pem

# Convert P7B to PEM
openssl pkcs7 -print_certs -in certificate.p7b -out certificate.cer

# Convert P7B to PFX
openssl pkcs7 -print_certs -in certificate.p7b -out certificate.cer
# OR
openssl pkcs12 -export -in certificate.cer -inkey privateKey.key -out certificate.pfx -certfile CACert.cer

# Convert PFX to PEM
openssl pkcs12 -in certificate.pfx -out certificate.p12 -nodes

# Convert PFX to P12
openssl pkcs12 -in certificate.pfx -out certificate.cer -nodes

# Extract Key
openssl pkcs12 -in C:\Users\rambo\Desktop\kimconnect_com.pfx -nocerts -out C:\Users\rambo\Desktop\key.pem -nodes

# Example:
PS C:\Program Files\OpenSSL\bin> .\openssl pkcs12 -in C:\Users\Test\Desktop\star_kimconnect_com.pfx -out C:\Users\Test\Desktop\star_kimconnect_com.cer -nodes
Enter Import Password:
# Done

Renew or Replace a SSL Certificate in Dynamics CRM

Error Message:
“Exchange Online Security Certificate Expiration Please update your certificate or Exchange Online integration will stop functioning in $count days.”

Resolution (steps):

1. Apply the new cert on the ADFS server
a. Obtain new cert and place it into C:\certs directory
b. Install new cert to local machine certificates store using MMC: Run certlm.msc > Personal > right-click Certificates > All Tasks > Import > Next > Browse > navigate to C:\certs > select the new cert > Open > Next > Next > OK > OK
c. Set cert access permissions: Run certlm.msc > Personal > Certificates > right-click new cert > All Tasks > Manage Private Keys > Add > search and select appropriate service accounts (‘AppPool user account’: READ, ‘ADFS service user account’: FULL) > OK > put a check mark next to appropriate permissions for each account > OK > OK
d. Make a backup of the old cert ***
e. Remove old cert from local machine certificates store using MMC: Run certlm.msc > Personal > Certificates > right-click on the old cert > Delete > Yes ***
f. Set Cert using AD FS Management Console:
– Run %windir%\ADFS\Microsoft.IdentityServer.msc: AD FS > Service > right-click Certificates > Set Service Communications Certificate > select the newly imported Cert > OK
– AD FS > Trust Relationship > Relying Party Trusts > right-click CRM Claims Relying Party > Update from Federation Metadata

2. Apply the new cert on Dynamics CRM Server’s IIS
a. Obtain new cert and place it into C:\certs directory
b. Remove old cert from local machine certificates store using MMC: Run certlm.msc > Personal > Certificates > right-click on the old cert > Delete > Yes
c. Install new cert to local machine certificates store using MMC: Run certlm.msc > Personal > right-click Certificates > All Tasks > Import > Next > Browse > navigate to C:\certs > select the new cert > Open > Next > Next > OK > OK >
d. Set cert access permissions: Run certlm.msc > Personal > Certificates > right-click new cert > All Tasks > Manage Private Keys > Add > search and select appropriate service accounts (‘AppPool user account’: READ, ‘ADFS service user account’: FULL) > OK > put a check mark next to appropriate permissions for each account > OK > OK
e. Apply new cert toward IIS: Run inetmgr.exe > Sites > Microsoft Dynamics CRM > click on Bindings on the right side panel > select https > Edit > click Select > highlight the newly imported cert > OK > OK > Close
f. Reset IIS: run iisreset

3. Apply new cert within CRM using Deployment Manager
a. Run “%PROGRAMFILES%\Microsoft Dynamics CRM\tools\Microsoft.Crm.DeploymentManager.exe” > Configure Claims-Based Authentication > Next > Next > Select > highlight the new Cert > OK > Next > Next > OK
b. Reset IIS: run iisreset

4. Run this script…