CrashPlan log categories

•July 1, 2014 • Leave a Comment

I’m a very happy customer of CrashPlan. Offsite backup is a critical component of any backup plan!

Without advanced¹ filesystems² like btrfs³, maintaining up-to-date backups is an arduous task. CrashPlan’s fire-and-forget nature lifts that weight from my shoulders, freeing my mind & time. Better yet, CrashPlan supports Windows & Linux.

However, like many large-scale cross-platform programs, it’s far from perfect. There are many cases where certain files fail to backup, where scanning for files slows the entire computer to a grinding halt, backups take longer than they should, file upload is not fully utilizing available bandwidth, or memory usage seems inordinate.

Fortunately, CrashPlan has a mature logging infrastructure. Code42 provides some insight on their website (mirror). If you investigate these logs, you’ll notice that they (a) are marked as a logging “level” (ERROR, WARN, INFO, DEBUG, TRACE, ALL, OFF), and (b) are categorized.

For (a), CrashPlan PROe “ADMINISTRATION CONSOLE COMMAND-LINE INTERFACE OVERVIEW“(mirror) suggests that the levels are actually [Error, Warn, Info, Fine, Trace], but I’ve never seen ‘Fine’ in the home edition.

For (b), the aforementioned document says only “The complete list of options is  available by contacting our Customer Champions.”.

Continue reading ‘CrashPlan log categories’

Goals: The Intended Outcomes of Higher Education

•June 27, 2014 • Leave a Comment

This chapter, written by Howard R. Bowen in “Foundations of American Higher Education” is a brilliant read.

Marx sought to change the world through changing social institutions, Jesus through changing the hearts of men. Higher education tries to do both.

Update: The Windows Phone app for WordPress makes no distinction between “save” and “post”. Here’s the chapter: Goals: The Intended Outcomes of Higher Education

Make VC++ Compiles Fast Through Parallel Compilation

•April 16, 2014 • Leave a Comment

Alexander Riccio:

Random ASCII always writes brilliant in-depth analyses!

Originally posted on Random ASCII:

The free lunch is over and our CPUs are not getting any faster so if you want faster builds then you have to do parallel builds. Visual Studio supports parallel compilation but it is poorly understood and often not even enabled.

I want to show how, on a humble four-core laptop, enabling parallel compilation can give an actual four-times build speed improvement. I will also show how to avoid some of the easy mistakes that can significantly reduce VC++ compile parallelism and throughput. And, as a geeky side-effect, I’ll explain some details of how VC++’s parallel compilation works.

Plus, pretty pictures.

View original 3,184 more words

“destroyed in a heartbeat”

•April 15, 2014 • Leave a Comment

I’ve recently stumbled across this slashdot article (mirrored)wherein the comments, MadX says:

*If* such a mechanism was coded in, the nature of open source would mean it would be found by others. This in turn would compromise the trust of the ENTIRE kernel. That trust can take years to build up – but be detroyed in a heartbeat.

Now that has a special irony.

Heartbleed?

“detroyed in a heartbeat”….or a heartbleed?

Arduino device driver trouble? A simple fix!

•April 13, 2014 • 2 Comments

Have you tried fixing an “Unknown device” error with your Arduino? If you’ve failed to resolve the issue, you likely saw a screen like this:

arduino_uno_driver

 

And maybe you even tried disabling driver signature enforcement as such:

disable_signature_enforcement

That is a very bad idea – and also unnecessary. Driver signature enforcement is a critical security feature, as a tremendous number of modern rootkits (and other malware) install drivers to do their dirty business.

Continue reading ‘Arduino device driver trouble? A simple fix!’

Rush post: heartbleed-masstest

•April 10, 2014 • Leave a Comment

The Heartbleed Bug (CVE-2014-0160) is not just a run-of-the-mill bug, it’s a damn scary bug. Most “nasty” vulnerabilities are quite limited in scope – maybe an attacker has a tiny chance of exploiting the vulnerability to execute (a tiny segment of) code, or they corrupt the appearance of files (ahem, winRAR) so the user is tricked into executing malicious code.  The heartbleed bug however, can be exploited without leaving any evidence of exploitation, and requires NO user interaction. The Heartbleed bug lets attackers read from arbitrary locations in the OpenSSL address space, including those used to store the PRIVATE keys.

 

As a result of this danger,  Mustafa Al-Bassam created a tool to scan websites for this vulnerability. I forked it on GitHub, and quickly hacked it to scan in a multithreaded fashion, much faster than the original serial method.

 

It’s really ugly at the moment, but you can clone it here: https://github.com/ariccio/heartbleed-masstest/

Malaysia Airlines Flight 370 …..and Paul Sajda?!?

•March 15, 2014 • Leave a Comment

In aid of the ongoing search for Malaysia Airlines Flight 370, millions of individuals are volunteering to screen satellite imagery for signs of the missing 777. Paul Sajda, whom I’ve written about before, has developed software that, with an EEG, allows an individual to screen hundreds of images per minute for “interesting” information. Maybe that software could be used to help search for Malaysia Airlines Flight 370?

That’s just an idea, some food for thought.

iMPCs: Cell Reprogrammers Take Aim at Liver Disease

•March 6, 2014 • Leave a Comment

Originally posted on NIH Director's Blog:

Cross-section of mouse liver

Caption: Cross-section of mouse liver containing iMPC-derived human liver cells (red), some of which are proliferating (green). All cell nuclei appear blue.
Credit: Milad Rezvani, Eli and Edythe Broad Center of Regeneration Medicine and Stem Cell Research, University of California, San Francisco

Over the past few years, researchers have learned how to reprogram skin or blood cells into induced pluripotent stem cells (iPSCs), which have the ability to differentiate into heart, nerve, muscle, and many other kinds of cells. But it’s proven a lot more tricky to coax iPSCs (as well as human embryonic stem cells) to differentiate into mature, fully functional liver cells.

Now, NIH-funded researchers at the University of California, San Francisco (UCSF) and the Gladstone Institutes appear to have overcome this problem. They have developed a protocol that transforms human skin cells into mature liver cells that not only function normally in a lab dish, but proliferate…

View original 685 more words

“The parameter is incorrect”: an update

•March 2, 2014 • Leave a Comment

I believe I’m zeroing in on the cause of the bug.

A comment in a mirror of a 1997 Sysinternals post, “Inside Windows NT Disk Defragmenting” says this about FSCTL_GET_VOLUME_BITMAP:

Return

If there are errors related to the volume’s support for this FSCTL or FileHandle representing a valid volume handle, an appropriate native NT error code is returned (as defined in the NT DDK file NTSTATUS.H). If the cluster specified in InputBuffer is out of range for the volume, the call will return STATUS_INVALID_PARAMETER. If there are no errors and there are no clusters beyond the last one described in the Map array, the FSCTL returns STATUS_SUCCESS. Otherwise STATUS_BUFFER_OVERFLOW is returned to notify the caller that further calls should be made to retrieve subsequent mappings.

…which is far more enlightening for this matter than Microsoft’s current documentation for FSCTL_GET_VOLUME_BITMAP:

Remarks

The FSCTL_GET_VOLUME_BITMAP control code retrieves a data structure that describes the allocation state of each cluster in the file system from the requested starting LCN to the last cluster on the volume. The bitmap uses one bit to represent each cluster:

  • The value 1 indicates that the cluster is allocated (in use).
  • The value 0 indicates that the cluster is not allocated (free).

Note that the bitmap represents a point in time, and can be incorrect as soon as it has been read if the volume has write activity. Thus, it is possible to attempt to move a cluster onto an allocated cluster in spite of a recent bitmap indicating that the cluster is unallocated. Programs using the DeviceIoControl function with the FSCTL_MOVE_FILE control code must be prepared for this possibility.

The handle used here must be a Volume handle and have been opened with any access. Note that only Administrators can open Volume handles.

The starting LCN in the input buffer may be rounded down before the bitmap is calculated. The rounding limit is file system dependent.

What exactly does Microsoft mean by “[...]the input buffer may be rounded down before the bitmap is calculated”?!!? I believe that that, together with the Sysinternals post, points to a bug in FSCTL_GET_VOLUME_BITMAP’s return behavior, or in defragsvc’s usage thereof.

To aid in debugging, I’m uploading a zip file with:

  • A Process Monitor trace of a defrag run wherein defrag throws the ”The parameter is incorrect. (0×80070057)”  error.
  • A dump of defrag.exe immediately after the error is thrown.
  • A dump of svchost.exe -defragsvc immediately after the error is thrown.
  • (hopefully informative set of) Screenshots of a WinDbg session immediately before, during, and after the error is thrown.
  • The symbols that I set breakpoints for in WinDbg.
  • Some remarks on FSCTL_GET_VOLUME_BITMAP collected from a few sources on the internet.
  • A few notes on the debugging process.

This slideshow requires JavaScript.

A note about the Process Monitor trace: I have removed a (very) few entries that contained personal information, entries that I’m quite certain have no relevance to the error.

Now it’s up to Microsoft to fix the issue.

DEFRAG_INVALID_PARAMETER_CENSORED

“The parameter is incorrect. (0x80070057)” and other Windows Defrag issues.

•February 26, 2014 • Leave a Comment

I’ve tried to defrag a 2TB hard drive for a couple of days now.

When defragging with /A /H /U /V /O, defrag nears completion, but throws a “The parameter is incorrect. (0x80070057)” error instead. This also happens when using the GUI, but defrag silently logs an error to the Event Log.

This slideshow requires JavaScript.

Another serious issue is the speed of defragmentation. As of right now (while consolidating free space), the defrag service spends the majority of it’s time eating 100% of a single core, and very little time on actual I/O. A quick bit of investigatory ProcMon work suggests a cause:

Is somebody incorrectly calling FSCTL_GET_RETRIEVAL_POINTERS?

Is somebody incorrectly calling FSCTL_GET_RETRIEVAL_POINTERS?

The only filter I’m using is:defrag4With that many BUFFER OVERFLOW events, there must be something wrong. Eventually, a Read Metadata DOES succeed, but that’s after several pages of BUFFER OVERFLOWs!

defrag5The Windows Dev Center gives us some insight:

Note that I've highlighted references to 'buffer'.

Note that I’ve highlighted references to ‘buffer’.

Note that I've highlighted references to 'buffer'.

Note that I’ve highlighted references to ‘buffer’.

If you really want to go deep, Mark Roddy has written a great article: Adventures in Luserland:  Finding Disk Sectors Associated with File Records.

Mirrored here: Adventures in Luserland

I’ll conjecture that Microsoft needs to review any defrag code related to FSCTL_GET_RETRIEVAL_POINTERS – but for now, I’ll just have to wait. And wait. And wait.

 
Andrzej's C++ blog

Guidelines and thoughts about C++

Bromium Labs

Call of the Wild Blog

fuzzing.info

the art of unexpected input engineering

Video Encoding & Streaming Technologies

Fabio Sonnati on video delivery and encoding

Freedom Embedded

Balau's technical blog on open hardware, free software and security

Paolo Bernardi

Paolo Bernardi, ramblings and notes (and Crypto-Gram for ebook readers)

The Embedded Code

Designing From Scratch

Bughira's Weblog

There is no such thing as closed source software...the processor sees every instruction, and so does the reverse engineer...

mov ah, 9<br>mov dx, hello_world_msg<br>int 21h

Just another WordPress.com weblog

Running the Gauntlet

Tank and Siko's Security Blog

mbrownnyc

so watch me do the funky dead butterfly.

The ThreatSTOP Blog

Stop Botnets Stealing from you

clevomods

home of the Custom light controller and LightFX library

Naked Security

Computer Security · News · Opinion · Advice · Research

root labs rdist

Embedded security, crypto, software protection

Biosingularity

Advances in biological systems.

Strategic Cyber LLC

A blog about Armitage, Cobalt Strike, and Red Teaming

Assumption Parish Police Jury

http://assumptionla.com/

Alexander Riccio

"Change the world or go home" -Microsoft Employee Slogan

Liquid Metals Project

Stuff that never made it into the paper.

We Are Made In NY

Learn, Launch and Find a Job in NYC Tech

Home Awesomation

It all started when I wanted to turn my fireplace on from my TV remote...

Mind Hacks

Neuroscience and psychology news and views.

tronixstuff

fun and learning with electronics

Cedar's Digest

Cognitive science, perception, teaching and ed reform

Walking Randomly

"Change the world or go home" -Microsoft Employee Slogan

"Change the world or go home" -Microsoft Employee Slogan

"Change the world or go home" -Microsoft Employee Slogan

"Change the world or go home" -Microsoft Employee Slogan

YouTube Blog

"Change the world or go home" -Microsoft Employee Slogan

Google Testing Blog

"Change the world or go home" -Microsoft Employee Slogan

Google Student Blog

"Change the world or go home" -Microsoft Employee Slogan

Google Research Blog

"Change the world or go home" -Microsoft Employee Slogan

"Change the world or go home" -Microsoft Employee Slogan

Google Public Policy Blog

"Change the world or go home" -Microsoft Employee Slogan

Google Open Source Blog

"Change the world or go home" -Microsoft Employee Slogan

Google Online Security Blog

"Change the world or go home" -Microsoft Employee Slogan

Google Enterprise Blog

"Change the world or go home" -Microsoft Employee Slogan

Gmail Blog

"Change the world or go home" -Microsoft Employee Slogan

Inside Search

"Change the world or go home" -Microsoft Employee Slogan

"Change the world or go home" -Microsoft Employee Slogan

"Change the world or go home" -Microsoft Employee Slogan

SearchReSearch

"Change the world or go home" -Microsoft Employee Slogan

Wikimedia blog

"Change the world or go home" -Microsoft Employee Slogan

Follow

Get every new post delivered to your Inbox.

Join 931 other followers