DHCP Leases Being Eaten - Cont...

So for the past week and a half I've had a very weird issue occurring where randomly all teh available DHCP leases in certain scopes would be eaten by *BAD_ADDRESS* followed by an invalid, 9 character MAC.

I explained how I thought I determined one of the culprits here: http://chronicgeekage.blogspot.com/2016/02/dhcp-leases-determine-what-device-is.html However it turns out this phone was not the issue.. it was actually a Lexmark printer.  I have no viewed the config yet but yesterday I discovered it after putting that unit back online.

sh cdp nei on my core switch revealed that the printer had somehow formed a routing loop.

sh cdp nei

sw4506

                 Gig 6/38          123             R S I  WS-C4506  Gig 2/26
sw4506

                 Gig 2/26          132             R S I  WS-C4506  Gig 6/38

Interface 6/38 was that printer.. forcing a shutdown then refreshing the DHCP leases page revealed my suspicions to be correct.

MDT 2013, PXE Boot Standalone Setup, BCD Issues

So this post is not about my entire MDT config but more focused on a recent issue that drove me nuts.  Many, many forums, blogs, websites out there about doing MDT installs without using WDS all point to using a tftp app like the free one from Solarwinds.

Which is fine, and very easy to setup and get going.  My pre-existing MDT server needed to be migrated to a physical and while doing so I decided to upgrade to Update 2 and the Windows 10 ADK.

In doing so meant that I'd have to re-do pretty much all of my configuration including the task sequences.  So I had PXE boot setup and working but was getting a blue error screen like this:

Hmm ok so in creating my BCD store file I used this batch script:

REM Creates BCD (boot configuration data) for MDT 2013 combined with WDS
REM —————————————————————————

REM CHANGE AS REQUIRED
REM ——————-
set BCD-File=C:\MDT\Boot\bcd
del %BCD-File%
Bcdedit /createstore %BCD-File%
Bcdedit /store %BCD-File% /create {ramdiskoptions}
Bcdedit /store %BCD-File% /set {ramdiskoptions} ramdisksdidevice boot
Bcdedit /store %BCD-File% /set {ramdiskoptions} ramdisksdipath \boot\boot.sdi

for /f “tokens=1-3” %%a in (‘Bcdedit /store %BCD-File% /create /d “WinPE x86” /application osloader’) do set guid1=%%c
Bcdedit /store %BCD-File% /set %guid1% systemroot \Windows
Bcdedit /store %BCD-File% /set %guid1% detecthal Yes
Bcdedit /store %BCD-File% /set %guid1% winpe Yes
Bcdedit /store %BCD-File% /set %guid1% osdevice ramdisk=[boot]\boot\LiteTouchPE_x86.wim,{ramdiskoptions}
Bcdedit /store %BCD-File% /set %guid1% device ramdisk=[boot]\boot\LiteTouchPE_x86.wim,{ramdiskoptions}

for /f “tokens=1-3” %%a in (‘Bcdedit /store %BCD-File% /create /d “WinPE x64” /application osloader’) do set guid2=%%c
Bcdedit /store %BCD-File% /set %guid2% systemroot \Windows
Bcdedit /store %BCD-File% /set %guid2% detecthal Yes
Bcdedit /store %BCD-File% /set %guid2% winpe Yes
Bcdedit /store %BCD-File% /set %guid2% osdevice ramdisk=[boot]\boot\LiteTouchPE_x64.wim,{ramdiskoptions}
Bcdedit /store %BCD-File% /set %guid2% device ramdisk=[boot]\boot\LiteTouchPE_x64.wim,{ramdiskoptions}

Bcdedit /store %BCD-File% /create {bootmgr} /d “WinPE x86”
Bcdedit /store %BCD-File% /set {bootmgr} timeout 30
Bcdedit /store %BCD-File% /set {bootmgr} displayorder %guid2% %guid1%
Bcdedit /store %BCD-File% /enum all

Which did not actually create a full, working BCD.  The syntax is off, it fails right after it creates the file and adds the ramdisk lines.

So my solution, manual creation.  I dropped to a CMD and manually ran each line in that batch leaving out the %guid1% and $guid2% parts and instead copying and pasting those manually.  I'll summarize was a ran below.

Bcdedit /createstore f:\tftp-root\boot\bcd
Bcdedit /store f:\tftp-root\boot\bcd /create {ramdiskoptions}
Bcdedit /store f:\tftp-root\boot\bcd /set {ramdiskoptions} ramdisksdidevice boot
Bcdedit /store f:\tftp-root\boot\bcd /set {ramdiskoptions} ramdisksdipath \boot\boot.sdi

Bcdedit /store f:\tftp-root\boot\bcd /create /d “WinPE x86” /application osloader

this line returns the first guid for you to record
Bcdedit /store f:\tftp-root\boot\bcd /set {insert guid1 here} systemroot \Windows
Bcdedit /store f:\tftp-root\boot\bcd /set {insert guid1 here} detecthal Yes
Bcdedit /store f:\tftp-root\boot\bcd /set {insert guid1 here} winpe Yes
Bcdedit /store f:\tftp-root\boot\bcd /set {insert guid1 here} osdevice ramdisk=[boot]\boot\LiteTouchPE_x86.wim,{ramdiskoptions}
Bcdedit /store f:\tftp-root\boot\bcd /set {insert guid1 here} device ramdisk=[boot]\boot\LiteTouchPE_x86.wim,{ramdiskoptions}

Bcdedit /store f:\tftp-root\boot\bcd /create /d “WinPE x64” /application osloader

The second guid to record
Bcdedit /store f:\tftp-root\boot\bcd /set {insert guid2 here} systemroot \Windows
Bcdedit /store f:\tftp-root\boot\bcd /set {insert guid2 here} detecthal Yes
Bcdedit /store f:\tftp-root\boot\bcd /set {insert guid2 here} winpe Yes
Bcdedit /store f:\tftp-root\boot\bcd /set {insert guid2 here} osdevice ramdisk=[boot]\boot\LiteTouchPE_x64.wim,{ramdiskoptions}
Bcdedit /store f:\tftp-root\boot\bcd /set {insert guid2 here} device ramdisk=[boot]\boot\LiteTouchPE_x64.wim,{ramdiskoptions}

Bcdedit /store f:\tftp-root\boot\bcd /create {bootmgr} /d “WinPE x86”
Bcdedit /store f:\tftp-root\boot\bcd /set {bootmgr} timeout 30
Bcdedit /store f:\tftp-root\boot\bcd {bootmgr} displayorder %guid2% %guid1%
Bcdedit /store f:\tftp-root\boot\bcd /enum all

Boom, done and working.

DHCP Leases, Determine what device is eating them up

Last Friday afternoon and again this morning I ran into an issue where all available DHCP leases for an entire scope were being used up by 'Bad Address'.  Turns out the device was also presenting an invalid MAC of:  a1020a, then a1020b, a1020c, etc, etc.

First I ping the first IP listed in DHCP to see if it's on the network.. and it was. Then I ssh'd into the core switch, the one performing Layer 3 functions.

sh ip arp

Resulted in a long list of recently resolved IP's, one of which was that first Bad Address. And it's giving me a valid MAC this time!


3750x>sh mac address-table address 0800.0f80.0b9d
          Mac Address Table
-------------------------------------------

Vlan    Mac Address       Type        Ports
----    -----------       --------    -----
   1    0800.0f80.0b9d    DYNAMIC     Gi1/1/4
Total Mac Addresses for this criterion: 1
3750x>sh int gi1/1/4
GigabitEthernet1/1/4 is up, line protocol is up (connected)
  Hardware is Gigabit Ethernet, address is 44d3.ca02.6834 (bia 44d3.ca02.6834)
  Description: Pole trendnet 10.2.1.35
  MTU 1500 bytes, BW 1000000 Kbit/sec, DLY 10 usec,
     reliability 255/255, txload 1/255, rxload 1/255
  Encapsulation ARPA, loopback not set
  Keepalive not set
  Full-duplex, 1000Mb/s, link type is auto, media type is 1000BaseSX SFP
  input flow-control is off, output flow-control is unsupported
  ARP type: ARPA, ARP Timeout 04:00:00
  Last input 00:00:06, output 00:00:00, output hang never
  Last clearing of "show interface" counters never
  Input queue: 0/75/0/0 (size/max/drops/flushes); Total output drops: 159
  Queueing strategy: fifo
  Output queue: 0/40 (size/max)
  5 minute input rate 182000 bits/sec, 156 packets/sec
  5 minute output rate 3720000 bits/sec, 382 packets/sec
     1373047180 packets input, 419190513395 bytes, 0 no buffer
     Received 64188296 broadcasts (55696255 multicasts)
     0 runts, 0 giants, 0 throttles
     0 input errors, 0 CRC, 0 frame, 0 overrun, 0 ignored
     0 watchdog, 55696255 multicast, 0 pause input
     0 input packets with dribble condition detected
     1971726846 packets output, 1221206759800 bytes, 0 underruns
 --More--

WOOT! Since I set meaningful descriptions on my switch ports I know where to go next.. The Pole switch.

SSH into 10.2.1.35

switch# sh mac-address-table address 08:00:0f:80:0b:9d

Vlan    Mac Address         Type     Ports
----    -----------         ----     -----
1       08:00:0f:80:0b:9d   Learnt   Gi0/18

Total Mac Addresses displayed: 1

switch#

Boom! Port 18.  I hit the switches Web gui and find out the port is actually hot, and is running at 100mb. 

Disable the port then translate the MAC address at http://www.macvendorlookup.com/

Now I play the waiting game.. hopefully a user will report that their phone is not working.

Windows 10, Helping to understand the new update strategy

So at work I'm looking into Windows 10 and how we can best deploy it (in the not-so-near future) to our user base.

Like many, I've been skeptical about the whole thing. The security issues you read about, wifi password sharing and the like.  We for the past few days I've been reading about the three different 'editions' and how they handle simple things like Windows Updates.

Microsoft believes that people, even in the enterprise environment, want new features and functionality faster than ever.  That product life-cycles should be measures in months and not years.  Not sure where that retarded idea came from but that's how it is now.

First off let's get this out, there are three editions now and they all center around how often the OS is updated with new features.  You have:

Current Branch (CB)
Current Branch for Business (CBB)
Long Term Servicing Branch (LTSB)

Windows 10 release types and cadences

Although Microsoft releases flight builds to Windows Insiders, Microsoft will publish two types of Windows 10 releases broadly to the public on an ongoing basis:

• Feature upgrades that install the latest new features, experiences, and capabilities on devices that are already running Windows 10. Because feature upgrades contain an entire copy of Windows, they are also what customers use to install Windows 10 on existing devices running Windows 7 or Windows 8.1, and on new devices where no operating system is installed.
• Servicing updates that focus on the installation of security fixes and other important updates.

They say to expect 2 - 3 Feature Upgrades per year... which to me sounds like 2 - 3 full installs per year.  But wait! there's more.

The cumulative nature of all Windows 10 releases

It is important to note that, in order to improve release quality and simplify deployments, all new releases that Microsoft publishes for Windows 10 will be cumulative. This means new feature upgrades and servicing updates will contain the payloads of all previous releases (in an optimized form to reduce storage and networking requirements), and installing the release on a device will bring it completely up to date. Also, unlike earlier versions of Windows, you cannot install a subset of the contents of a Windows 10 servicing update. For example, if a servicing update contains fixes for three security vulnerabilities and one reliability issue, deploying the update will result in the installation of all four fixes.  

Great, just great.  Now every Feature Upgrade really does contain the sum of every other update.  Microsoft believe that every person out there has a good fat pipe to the internet, that 'It's 2016! 10mb+ broadband is everywhere!' At least that's my opinion anyway.

Don't stop reading yet, there's more news for the enterprise folks.

Windows Update use within enterprises

Although Windows Update greatly simplifies and accelerates update deployment, enterprises are not using Windows Update as broadly as consumers and small businesses. This is largely because Windows Update maintains control over which updates are installed and the timing of installation. This makes it difficult for IT administrators to test updates before deployment in their specific environment.

The role of Windows Server Update Services

To help address the concerns of IT administrators, Microsoft released Windows Server Update Services in 2005. Windows Server Update Services enables IT administrators to obtain the updates that Windows Update determines are applicable to the devices in their enterprise, perform additional testing and evaluation on the updates, and select the updates they want to install. Windows Server Update Services also provides IT administrators with an all or nothing way to specify when they want an approved update to be installed. Because IT administrators ultimately select and install most updates identified by Windows Update, the role of Windows Server Update Services in many enterprises is to provide IT administrators with the additional time they need to gain confidence in the quality of updates prior to deployment.

New Windows Update capabilities in Windows 10

To enable enterprises to manage more of their devices using Windows Update directly, Windows 10 provides IT administrators with a way to configure devices so that Windows Update will defer new feature upgrade installations until approximately four months after Microsoft first publishes them. The additional time can be used to perform testing or enable releases to gain additional time in market prior to deployment.

At the end of each approximately four month period, Microsoft executes a set of processes that require no action from enterprise IT administrators.* First, Microsoft creates new installation media for the feature upgrade by combining the original installation media with all the servicing updates published by Microsoft since the original media’s release. This reduces the time it can take to install a feature upgrade on a device. Second, Microsoft republishes the new media to Windows Update withtargeting instructions that state (in effect) “install this media on devices that are configured for deferred installation of new feature upgrades.” At this point, devices configured to defer installation will begin receiving and installing the feature upgrade automatically.

The role of Windows Update for Business

Although Windows 10 will enable IT administrators to defer installation of new feature upgrades using Windows Update, enterprises may also want additional control over how and when Windows Update installs releases. With this need in mind, Microsoft announced Windows Update for Business in May of 2015. Microsoft designed Windows Update for Business to provide IT administrators with additional Windows Update-centric management capabilities, such as the ability to deploy updates to groups of devices and to define maintenance windows for installing releases. This article will be updated with additional information about the role of Windows Update for Business in servicing Windows 10 devices as it becomes available.

* This is what I really enjoy as an SA... control being taken away.  

Entire article here:  https://technet.microsoft.com/en-us/library/mt598226(v=vs.85).aspx

Granted according this article, which appears to be a constantly updated piece, is that the way things are right now I can maintain control over things like this by going with the LTSB release.

More to come.

Ubuntu 12.04 LTS - Mount terminated with status 32

I have a cloned Ubuntu 12.04 vm (ESXi 5.5) that was giving the below error while mounting an NFS share:

mount.nfs: Connection timed out
mountall: mount /netapp [514] terminated with status 32

/etc/fstab:

cat /etc/fstab
netapp:/vol/vol0/custom /netapp nfs intr 0 0



After much time spent Googling I discovered that a number of other folks had resolved this issue by either installing or replacing their GUI.  Following another advise I installed KDM, then shortly after I remembered by dislike for KDE so I replaced it with GDM.

Some might argue the mount issue can't be related to a GUI, to that I can't argue.  But I know what resolved my error.  For good measure though after the fact I altered my mount line to:

netapp:/vol/vol0/custom /netapp nfs intr,_netdev,timeo=15,retrans=7  0 0

Powershell - Inactive computer accounts, 90 days

A little script I wrote a while back, it will return to the screen, all computer objects that have a LastLogonTimestamp of older than 90 days.


# Calculate the UTC time 90 days ago, in FileTime (Integer) format and convert it to a string
$LLTSlimit = (Get-Date).AddDays(-90).ToFileTimeUTC().ToString()
# Create the LDAP filter for the AD query
# Searching for enabled computer accounts which have lastLogonTimestamp older than 90 days
$LDAPFilter = "(&(objectCategory=Computer)(lastlogontimestamp<=$LLTSlimit) (!(userAccountControl:1.2.840.113556.1.4.803:=2)))"
# Create an ADSI Searcher to query AD
$Searcher = new-object DirectoryServices.DirectorySearcher([ADSI]"")
$Searcher.filter = $LDAPFilter
# Execute the query
$Accounts = $Searcher.FindAll()
# Process the results
If ($Accounts.Count –gt 0) {
# Create an array to store all the results
$Results = @()

# Loop through each account

ForEach ($Account in $Accounts) {

# Create an object to store this account in
$Result = "" | Select-Object Name,ADSPath,lastLogonTimestamp
# Add the name to the object as a string
$Result.Name = [String]$Account.Properties.name
# Add the ADSPath to the object as a string
$Result.ADSPath = [String]$Account.Properties.adspath
# Add the lastLogonTimestamp to the object as a readable date
$Result.lastLogonTimestamp = `

[DateTime]::FromFileTime([Int64]::Parse($Account.Properties.lastlogontimestamp))

# Add this object to our array

$Results = $Results + $Result
}
}

# Output the results
$Results | Format-Table -autosize
# Extending this script to disable the discovered accounts is as easy as adding this code snippet to the end:
# Disable each account
#ForEach ($Result in $Results) {
#$ADSIAccount = [ADSI]$Result.ADSPath
#$ADSIAccount.PSBase.InvokeSet("AccountDisabled", "True")
#$ADSIAccount.SetInfo()
#}