File Toolkit



Engagement Toolkit for Managers and Leaders This toolkit includes best practices from the Corporate Leadership Council (CLC)1. And supplemental information. Firstly, the ToolKit can be used to download classes in separated folders. The argument -classes accepts a list of classes or the path to the file.txt (-classes path/to/file.txt) that contains the list of all classes one for each lines (classes.txt uploaded as example). The above notebook has been constructed as an example that creates a CSV file that can be used within Matlab. Gridded Atmospheric WIND Toolkit This 50-TB subset contains instantaneous 1-hour resolution model output data for 7 years on a uniform 2-km grid that covers the continental U.S., the Baja Peninsula, and parts of the Pacific and Atlantic.

  1. File Repair Toolkit
  2. Toolkit File Formats
  3. Rar File Toolkit
  4. Toolkit File Backup

If you are a system administrator, IT professional, or a power user it is common to find yourself using the command prompt to perform administrative tasks in Windows. Whether it be copying files, accessing the Registry, searching for files, or modifying disk partitions, command-line tools can be faster and more powerful than their graphical alternatives. This tutorial will walk you through creating a command-line toolkit that contains useful programs and utilities that can make administering and using your computer easier and more efficient. The tutorial will also walk you through configuring your PATH environment variable so that these tools are available whenever you need them without having to specify the complete path to your toolkit folder. At the end of the tutorial we have listed a variety of command-line programs that are included with Windows or are by 3rd party developers that you can use as part of your command-line toolkit.

The first step is to create a folder that you will use to store your command-line programs. This folder can be located anywhere, but should have a name that describes what it is being used for. Some example folder names that you can use are bin, cl, or command-line. For the purpose of this tutorial, we will use the folder C:command-line to store the command-line tools that we would like to use. Once the folder has been created, we now want to add it to the Windows PATH so that we do not have to type the full path to the command-line tool every time we wish to use one.

To do this, click on the Start button and type System. If you are using Windows 8, you can just type System from the Start Screen. When the search results appear, click on the System control panel in the search results to open the control panel as shown below.

Now click on the Advanced system settings option as indicated by the red arrow in the image above. This will open the Advanced tab for the System Properties screen.

Now click on the Environment Variables button to open a screen that lists the various environment variables that are configured in Windows.

Under the System variables box scroll down till you see the Path variable. Once you see that variable, double-click on it to open a screen where you can edit it.

The Path variable is a list of folders separated by a semi-colon (;) that Windows will use to search for programs to execute when you type them in. When you try to launch a program from the command-line, Windows will search through all the folders in its path and execute the program if it is found. As we do not want to have to type the full path to a command-line program (C:command-lineprogram.exe) every time we use it, we can add the C:Command-line folder to our path so we only have to type the program name (program.exe) to launch it.

As our command-line tools in this tutorial are located in C:command-line we want to add this folder to the end of the list of folders that are already present in the Variable value field. To do this, go to the very end of the text in the Variable value field and type ;C:command-line. When you do this you will need to substitute C:command-line with the path to your folder. When you are done, you should now see the field that looks similar to the image above.

To save your changes, click on the OK button and then close the System Control Panel. Now whenever you type in a program name that is stored in your command-line program folder, Windows will be able to find it and execute it.

Useful Command-line programs to add to your command-line toolkit

This section will list a variety of command-line programs that can you use to start your toolkit. When using the list below, if the program is not bundled with Windows, then the name of the program will also be a link to the site that you can use to download the program and save it to your command-line folder. If the program name does not contain a link, then it is bundled with Windows and can already be used from your command prompt. If there are any other tools that you recommend we add to this list, please let us know.

Administration and Troubleshooting Programs

Command Description
AccessChkAccessChk lists the kind of permissions specific users or groups have to resources including files, directories, Registry keys, global objects and Windows services
atThe AT command schedules commands and programs to run on a computer at a specified time and date. The Schedule service must be running to use the AT command.
CoreInfoCoreinfo is a command-line utility that shows you the mapping between logical processors and the physical processor, NUMA node, and socket on which they reside, as well as the cache’s assigned to each logical processor.
driverqueryDisplays a list of installed device drivers.
MpCmdRun.exeA command-line interface for Windows Defender. To execute this program you must use the full path: %ProgramFiles%Windows DefenderMpCmdRun.exe
netVarious Windows management commands. More information can be found here.
netshNetsh is a command-line scripting utility that allows you to, either locally or remotely, display or modify the network configuration of a computer that is currently running. More information can be found here.
powershellWindows PowerShell is a task-based command-line shell and scripting language designed especially for system administration. More information can be found here.
PsLogListAllows you to list the contents of local or remote computer's Windows Event Log.
PsPasswdPsPasswd is a tool that lets you change an account password on the local or remote systems.
PsServiceAllows you to list and configure Windows services.
runasRun a program as another user.
rundll32Execute functions exported in a DLL file.
scManage Windows Services.
shutdownShutdown a local or remote computer.
SigCheckVerify that images are digitally signed and dumps version information contained within the file.
UnixUtilsA collection of Unix utilities that have been ported to Windows. These utilities are very useful and include programs like grep, split, tar, dir, etc.
wmicA program that allows command-line and batch file access to Windows Management Instrumentation. More information can be found here.
WUInstallA command-line Windows Update installer and management program.

Boot and Windows Startup Programs

Command Description
bcdbootThe bcdboot.exe command-line tool is used to copy critical boot files to the system partition and to create a new system BCD store. More information can be found here.
bcdeditThe Bcdedit.exe command-line tool modifies the boot configuration data store. The boot configuration data store contains boot configuration parameters and controls how the operating system is booted. This tool is for Windows Vista and later. More information can be found here.
bootcfgMore information can be found here.
repair-bdeThe bootcfg command is a Microsoft Windows Server 2003 utility that modifies the Boot.ini file. This command has a function that can scan your computer's hard disks for Microsoft Windows NT, Microsoft Windows 2000, Microsoft Windows XP, and Windows Server 2003 installations, and then add them to an existing Boot.ini file or rebuild a new Boot.ini file if one does not exist. You can use the bootcfg command to add additional Boot.ini file parameters to existing or new entries. More information can be found here.

File Comparison, Search, and Viewing Programs

Command Description
compCompares the contents of two files or sets of files.
findstrSearches for strings in files. This is a powerful tool, but contains a limited Regular Expression functionality. If you want a string searching tool with greater RegExp functionality, you may want to use grep that is part of the UnixUtils package.
fcCompares two files or sets of files and displays the differences between them.
moreDisplays a file one page at a time.
sortReads input, sorts data, and writes the results to the screen, to a file, or to another device. More information about sort can be found here.
typeDisplays the entire file to the screen.

File Permission and Management Programs

Command Description
7ZipFull featured archive program that can work with almost any archive type. When adding this to your command-line folder, be sure to copy both 7z.exe & 7z.dll for it to work properly.
attribDisplays, sets, or removes the read-only, archive, system, and hidden attributes assigned to files or directories. Used without parameters, attrib displays attributes of all files in the current directory. More information can be found here.
cdChanges the current working directory.
copyCopy a file to another name or to a different folder.
dirList the files in a folder.
File Checksum Integrity VerifierThe File Checksum Integrity Verifier (FCIV) utility can generate MD5 or SHA-1 hash values for files to compare the values against a known good value. FCIV can compare hash values to make sure that the files have not been changed.
forfilesSelects a file (or set of files) and executes a command on that file.
HandleHandle is a utility that displays information about open handles for any process in the system. You can use it to see the programs that have a file open, or to see the object types and names of all the handles of a program.
icaclsDisplays or modifies discretionary access control lists (DACLs) on specified files, and applies stored DACLs to files in specified directories. More information about icacls can be found here.
JunctionAllows you to create, list, or delete Junctions in Windows.
LADSLADS will display a list of all alternate data streams found in a particular folder.
md5sumLists the md5 has for a particular file or numerous files in a folder.
moveMove a file or folder to another location.
renRename a file or folder.
SdeleteYou can use SDelete both to securely delete existing files, as well as to securely erase any file data that exists in the unallocated portions of a disk (including files that you have already deleted or encrypted). SDelete implements the Department of Defense clearing and sanitizing standard DOD 5220.22-M, to give you confidence that once deleted with SDelete, your file data is gone forever.
sfcScans the integrity of all protected system files and replaces incorrect versions with correct Microsoft versions.
StringsDisplays strings found within a file.
xcopyCopies files and directories, including subdirectories.

Filesystem Management Programs

Command Description
chkdskChecks a disk and displays a status report.
defragLocates and consolidates fragmented files on local volumes to improve system performance.
diskpartDiskpart allows you to manage and modify disk partitions. More information about diskpart can be found here.
FixMBRRepairs the master boot record of the boot disk. The fixmbr command is only available when you are using the Recovery Console.
recoverRecovers readable information from a bad or defective disk.
takeownThis tool allows an administrator to recover access to a file that was denied by re-assigning file ownership.
File

Network Diagnostics & Administration Programs

Command Description
arpDisplays and modifies the IP-to-Physical address translation tables used by address resolution protocol (ARP). Useful for finding mac addresses of other networked devices on your network.
cURLcURL is a command line tool for downloading web pages, entire sites, ftp files, etc.
ipconfigDisplays all current TCP/IP network configuration values and refreshes Dynamic Host Configuration Protocol (DHCP) and Domain Name System (DNS) settings. Used without parameters, ipconfig displays the IP address, subnet mask, and default gateway for all adapters. More information can be found here.
NetcatNetcat is a featured networking utility which reads and writes data across network connections, using the TCP/IP protocol. This is a very useful tool for diagnosing network connections, open firewall ports, or for sending the output of a local command to a remote computer.
netstatDisplays protocol statistics and current TCP/IP network connections.
NmapNmap ('Network Mapper') is a utility for network discovery and security auditing. This program can quickly perform a TCP/IP audit of your network.
nslookupNslookup allows you to perform DNS (Domain Name Service) resolution.
pathpingThe PathPing tool is a route tracing tool that combines features of Ping and Tracert with additional information that neither of those tools provides. PathPing sends packets to each router on the way to a final destination over a period of time, and then computes results based on the packets returned from each hop. Since PathPing shows the degree of packet loss at any given router or link, you can pinpoint which routers or links might be causing network problems. More information can be found here.
pingPing is a computer network administration utility used to test if you can reach a host on an Internet Protocol (IP) network and to measure the round-trip time for messages sent from the originating host to a destination computer.
PsFilePsFile is a command-line utility that shows a list of files on a system that are opened remotely, and it also allows you to close opened files either by name or by a file identifier.
PsExecPsExec is a program that lets you execute processes on other systems, complete with full interactive use for console applications, without having to manually install client software. Please note that some anti-virus vendors may detect this as 'Remote Admin', but it is a legitimate tool from Microsoft.
PsLoggedOnPsLoggedOn is an program that displays both the locally logged on users and users logged on via resources for either the local computer, or a remote one. If you specify a user name instead of a computer, PsLoggedOn searches the computers in the network neighborhood and tells you if the user is currently logged on.
routeDisplays and modifies the entries in the local IP routing table. Used without parameters, route displays help. More information can be found here.
tracertDisplays the path taken from TCP/IP packets as they traverse from your local computer to a remote target. More information can be found here.
WgetGNU Wget is a program for retrieving files using HTTP, HTTPS and FTP, the most widely-used Internet protocols.

Process Management Programs

Command Description
ListDllsListDLLs is a utility that reports the DLLs loaded into processes. You can use it to list all DLLs loaded into all processes, into a specific process, or to list the processes that have a particular DLL loaded.
PsKillAllows you to terminate processes.
PsListLists all running processes.
tasklistLists all running running processes and services. This program can also be used to list what services are running under a particular svchost process. See here for more information regarding how to do that.
taskkillThis tool is used to terminate tasks by process id (PID) or image name.

If there are any other command-line tools that you think we missed, please let us know about them.

The Wind Integration National Dataset (WIND) Toolkit is an update and expansion of the Eastern Wind Integration Data Set and Western Wind Integration Data Set. It supports the next generation of wind integration studies.

WIND Toolkit Resources

The Wind Integration National Dataset (WIND) Toolkit in the NREL Wind Prospector

The Wind Integration National Dataset (WIND) Toolkit, Applied Energy (2015)

Overview and Meteorological Validation of the Wind Integration National Dataset Toolkit, NREL Technical Report (2015)

Validation of Power Output for WIND Toolkit Data, NREL Technical Report (2014)

Wind Toolkit Forum, Ask questions, get information, share tips

File Repair Toolkit

The WIND Toolkit includes instantaneous meteorological conditions from computer model output and calculated turbine power for more than 126,000 sites in the continental United States for the years 2007–2013. It features three data sets:

  • The meteorological data set includes basic information on the weather conditions in each 2-km x 2-km grid cell. The meteorological data set also includes parameters such as wind profiles, atmospheric stability, and solar radiation data in those cells. This complete data set is not yet publicly available, though some of the data is made available with the power data set.
  • The power data set was created using the wind data at 100-meter hub height and site-appropriate turbine power curves to estimate the power produced at each of the turbine sites.
  • The forecast data set includes forecasts for 1-hour, 4-hour, 6-hour, and 24-hour forecast horizons.

The WIND Toolkit has been funded by the U.S. Department of Energy, Office of Energy Efficiency and Renewable Energy, Wind and Water Power Technologies Office and was created through the collaborative efforts of NREL and 3TIER.

Access the Data

Because of the size of the full WIND Toolkit, only the following data subsets are available for user access and download. These sets are tailored to the needs of common applications.

Techno-Economic Summary and Index

This 10-MB subset contains summarized statistics for 120,000 points within the continental U.S. selected for their wind resource. It can be downloaded directly through the NREL Data Catalog.

Offshore Summary Dataset

This dataset contains summary statistics for offshore wind resources (Atlantic, Pacific, and Gulf) for the continental United States derived from the WIND Toolkit. These statistical summaries are aligned to actual offshore lease blocks and were developed under contract by NREL for the Bureau of Oceanic Energy Management.

For more information and to download this data, visit the WIND Toolkit Offshore Summary Dataset on Data.gov.

Techno-Economic WIND Toolkit

This 2-TB subset contains instantaneous 5-minute resolution model output data for 7 years at 120,000 points within the continental U.S. selected for their wind resource. This set contains power estimates along with a subset of atmospheric variables (e.g., 100-meter wind speed). Aligned forecast data are also available at 1-hour resolution for 1, 4, 6, and 24 hours ahead.

For small areas and individual points:

  1. Access the Wind Toolkit via the Wind Prospector interface.
  2. Zoom to the location you are interested in, and choose a data point.
  3. Select the Analysis & Downloads tab at the top of the left column.
  4. Select the box for Wind Resource Data Download (Point), and click on the point you want.
  5. Fill in the Data Download Information Form.
  6. Fill in the Data Download Wizard, and select the years, attributes, and options you want. Watch the download indicator at the bottom; it should be blue. If it's not, you'll need to reduce the duration.
  7. A link to the data download will arrive by email.
  8. Click the link to download the data.

For slightly larger areas, download the data through Globus.

  1. Follow the instructions above to identify the area and period you want.
  2. Select the locations, temporal range, and attributes using the draw tool.
  3. An email will be sent indicating that the data is ready for download.
  4. Follow the instructions below to download the data.

See full instructions for Wind Toolkit data downloads via API from NREL's Developer Network.

  1. Download the summary and index through the NREL Data Catalog.
  2. Download the power data through Globus.
  3. Download the forecast data through Globus.

The PyWTK library provides a python interface and API for techno-economic data. This library can access the data locally once transferred from S3 or via Amazon Web Services (AWS) Lambda using data in AWS S3 directly.

To use AWS Lambda, start by cloning the pywtk repo:

git clone https://github.com/nrel/pywtk

Install Anaconda python (https://www.continuum.io/downloads) and the required packages and start a pywtk environment.

In pywtk directory:

conda env update -n pywtk
source activate pywtk
python setup.py install

Start a jupyter notebook with:

jupyter notebook --no-browser

Toolkit file formats

In your browser, you can paste the output URL from Jupyter notebook and browse to the site_retrieval_lambda.ipynb notebook.

This will give you an interface for programming python and downloading site data. The above notebook has been constructed as an example that creates a CSV file that can be used within Matlab.

Gridded Atmospheric WIND Toolkit

This 50-TB subset contains instantaneous 1-hour resolution model output data for 7 years on a uniform 2-km grid that covers the continental U.S., the Baja Peninsula, and parts of the Pacific and Atlantic oceans. NREL has made the entire Gridded Atmospheric WIND Toolkit available using an API and data hosted using Amazon cloud services.

The Highly Scalable Data Service (HSDS) provides access to the entire 50-TB gridded data with a RESTful JSON API and supporting Python library developed to deliver the data to users in a way that is similar to accessing data locally with the HDF5 library.

To get started, example notebooks that demonstrate how to access the data are available at https://github.com/NREL/hsds-examples.

Server

You can make a local copy (clone) of this repository using git:

git clone https://github.com/NREL/hsds-examples

An interactive visualization allows users to explore the entire 7 years of wind toolkit data at any resolution.

The source code behind this application is available for reference for those who may wish to develop their own web-based applications that use the HSDS API.

Processed Weather Modeling (WRF) Outputs

This 0.5-PB subset contains instantaneous 5-minute resolution model output data for 7 years on a uniform 2-km grid. Because of the size of this dataset, it is not publicly available at this time. Researchers who need full-resolution data from the WIND Toolkit should submit a detailed request using the contact information below.

Validate the Data

Users of the WIND Toolkit need to know the information presented is accurate and that errors are known so corrective steps can be taken; therefore, a validation code written in R is provided to allow users to validate data at their own locations. Validation is based on statistical analyses of wind speed using error metrics such as bias, root mean squared error, centered root mean squared error, mean absolute error, and percent error. Additionally, plots of diurnal cycles, annual cycles, wind roses, histograms of wind speed, and quantile-quantile plots are created to visualize how well observational data compare to model data. Ideally, validation will confirm beneficial locations to use wind energy and encourage regional wind integration studies using the WIND Toolkit.

Download these validation code files:

  • Functions.

Toolkit File Formats

For more information, see A Guide to Using the WIND Toolkit Validation Code.

Cite the Data

Users of the WIND Toolkit should use the following citations:

Draxl, C., B.M. Hodge, A. Clifton, and J. McCaa. 2015. Overview and Meteorological Validation of the Wind Integration National Dataset Toolkit (Technical Report, NREL/TP-5000-61740). Golden, CO: National Renewable Energy Laboratory.

Draxl, C., B.M. Hodge, A. Clifton, and J. McCaa. 2015. 'The Wind Integration National Dataset (WIND) Toolkit.' Applied Energy 151: 355366.

Lieberman-Cribbin, W., C. Draxl, and A. Clifton. 2014. Guide to Using the WIND Toolkit Validation Code (Technical Report, NREL/TP-5000-62595). Golden, CO: National Renewable Energy Laboratory.

Rar File Toolkit

King, J., A. Clifton, and B.M. Hodge. 2014. Validation of Power Output for the WIND Toolkit (Technical Report, NREL/TP-5D00-61714). Golden, CO: National Renewable Energy Laboratory.

Contact

The WIND Toolkit Team

Toolkit File Backup

WINDtoolkit@nrel.gov | Wind Toolkit Forum