1. Load balancing disk usage when uploading a large amount of files of considerable individual file size. |
The following code uses a number of techniques to manipulate files and directories. This example deals with the situation whereby a large volume of files are being uploaded to a server of considerable individual file size. Three hard disk drives(120 GB each and can easily add more) are used on a separate machine (windows™ specific paths) to cope with this situation and this code detects how full these disks are and uploads files appropriately so that used disk space for each disk does not exceed approximately 80%. An XloadCounterFileRename object is used to arbitrarily rename each file to an arbitrary number. All MIME types are being allowed and a blank file field is not. Also, the default maximum file size of 1GB is being employed.
XloadManager xman = new XloadManager(request, new XloadCounterFileRename());
//decide optimum destination for files. String[] paths = new String[3]; paths[0] = "\\\\computername\\sharedfolderDISKA\\files"; paths[1] = "\\\\computername\\sharedfolderDISKB\\files"; paths[2] = "\\\\computername\\sharedfolderDISKC\\files"; int maxSizeOnDisk = 96 * 1024; //80% of drive space considered max. XloadDirectory dir = null; String target = null; for(int i = 0; i < paths.length; i++){ dir = xman.getDirectory(paths[i]); float mb = dir.getMegabyteSize(true); if(mb < maxSizeOnDisk)target = paths[i]; } if(target == null){ //return a response stating that all available space is used. } //end of decision making process for destination directory.
xman.target("file1", target); xman.upload();
StringBuffer error = new StringBuffer(100); XloadFileUpload upload = (XloadFileUpload)xman.getFileUpload("file1"); if(upload.isAnUploadFailure()){ error.append("Your file failed to upload due to:" + //list possible reasons or give generic //message. "\n"); }else{ XloadFile file = upload.getFile(target); String originalNameOnClient = file.getRemoteName(); String nameOnDisk = file.getWrittenName(); //place file details inside relational database. }
//return a response using the error object if needed.
|
Note that no chronological sequence of numbers is guaranteed within a directory but that each file will have a unique name within a directory, including the file extension. Also note that we decide upon a target directory before the upload() method is called and certain methods are available within the XloadManager class to do this (i.e. getMegabyteSize()).
This code can obviously be optimized as upon each upload the getMegabyteSize() method is called relatively unintelligently when a flag system could be used to better effect and a better load balancing algorithm could be employed; but it shows the principle very adequately and is a starting point for a production system. What is clear is that within only a few lines a complex situation is dealt with.
Note the use of network paths for the locations of the hard disks. Xload can seamlessly write, copy or move files to network locations.
© Gubutech(Xload) 2006 (v1.2)