Java library for OpenStack Storage, aka Swift

View project onGitHub

Release notes

v0.10.2

v0.10.1

v0.10.0

v0.9.17

v0.9.16

v0.9.8

v0.9.7

v0.9.6

  • Absolute paths for mock fileset; up till now, the only path that would be accepted would be a relative one. This would very likely translate to a target/classes folder when run locally. With the new setup, one can set setMockOnFileObjectStoreIsAbsolutePath to true and pass an absolute path. In a web container, this would very likely be a path determined with ServletContext.getRealPath().

v0.9.5

  • Pseudo-hierarchy delimiters; in order to read Swift's pseudo-hierarchies as you would directories (that is, if you list within a folder, you will only see the contents of that folder, not the ones nested within, as would be the default for Swift's list commands). By either explicitly passing a delimiter (most likely the '/' character), or making use of JOSS' built-in listDirectory commands, you can make Swift treat your folders as real folders.
  • Custom headers; if JOSS is required to pass extra, custom headers to Swift, which do not fit the standard pattern, you can now add these to the Container. JOSS will then take care to set these headers on the create request for a container.
  • Flexible Basic / TempAuth username determination; when both username and tenantname are passed, one username will be drafted with a colon as separator. If only the username is passed, only that will be used, assuming the user has already composed it.

v0.9.4

  • TempAuth; TempAuth is a custom authentication module for Swift. It can be configured in JOSS by setting the authentication mechanism to TEMPAUTH.

v0.9.3

  • Fixed URL encoding bug; Java's default URLEncoder codifies spaces as '+', whereas %20 codes are required. An encoding call to URLEncoder is now supplemented with a replace("+","%20"). More nefariously, URLEncoder.encode was also called on the TempURL plaintext signature. This caused matching errors on temporary GET/PUT urls, because Swift uses the pure name (including spaces and unicode characters).
  • Basic authentication; Before, JOSS only supported Keystone authentication. Now, you can also use Basic authentication. Call .setAuthenticationMethod(AuthenticationMethod.BASIC) on the AccountFactory and make sure to pass a username/password. Note that Swift does not require a tenant for this type of authentication.

v0.9.2

v0.9.1

  • FormPost; JOSS generates a signature which can be used to upload a file directly from the browser using a regular form element

v0.9.0

  • Static Website; Static Website is a child of Container and allows you to convert a Container into a website. You can set the index and error page and choose to choose the site as a web listing (including CSS reference). Upload and download of entire folders including pseudo-hierarchies are fully supported.
  • set/get single Metadata key/value pairs; a single metadata key/value pair can now be stored or fetched. To allow for economization, JOSS also has methods that don't place the save call immediately, but instead wait for the explicit saveMetadata() call.
  • Websites always public; Whenever a Static Website is created, it is immediately made public
  • Website host; A Static Website can be asked what its host is. This assumes that domainremap (ie, container becomes part of the host and is removed from the URL path) is used by the implementor. The host can be configured as a single CNAME entry, so that the container is not part of the URL path
  • Ignore files; For static website pushing/pulling, you can enter a number of file/directory masks that must be ignored.
  • Tenant ID; Trystack.org uses tenant ID more than name. The configuration supports both or one of either.
  • Lenient Access object; authentication now longer fails when new fields have been added to the Access object. These new fields are ignored
  • Auto-discover tenants; Trystack.org supplies username/password, but not the tenant. There is a process where you can find your tenant by checking out the Access file manually. JOSS does this for you. If no tenant is supplied and it finds exactly one, it will use that one. A warning will be logged, though.
  • Refactor preferred region; Preferred region can be configured in AccountConfig.
  • TempURL for private content; objects in private container can be accessed directly using the TempURL concept. A URL is made consisting of a signature and expiry. The server checks these for validity before granting access
  • Ambiguous service catalog elements; Folsom has a feature/bug that make an empty service catalog be different from a filled one, with regards to being either an array or a map. The situation can be traced to a tenant having been passed or not. Both situations now have their own Access object to deal with this situation
  • Metadata key capitalized; Swift automatically converts underscores to dashes for metadata keys and makes the first letter for each word upper case. The Mock now mimicks this behaviour as well
  • Synchronize time; local and server time will very likely be different. JOSS now checks the server time, compares it to its own and determines a time modifier. The modifier is used for calculating the TempURL signature and expiry time
  • Verify TempURL; an object can verify a signature and expiry time. This is useful if the application archictecture will have to deal with both real and mock implementations
  • Saving metadata refreshes; whenever metadata is saved, the next read order with automatically fetch all metadata.
  • Mock keeps non-passed metadata; pre-Folsom Swift deleted metadata that was not explicitly passed. Mock is now updated to mimick this fix and no longer deletes non-passed metadata

v0.8.6

  • Account keeps track of Container instances; with the introduction of getURL(), a potential economy issue arises. Because getURL() wants to know whether a Container is public or private, a HEAD call to Container will have to be made. With this release, Account will keep track of its Container instances (ie, cache) and reuse them. This will minimize the number of server calls. Container Cache is enabled by default.

v0.8.5

  • getPublicURL(), getPrivateURL(), getURL(); you can now pass publicHost and privateHost in your configuration. Note that publicHost replaces host (**API change!**). getPrivateURL() works in a similar manner to getPublicURL(), except it takes privateHost as its host. A new player is getURL(), which decides to either give back getPublicURL() or getPrivateURL() based on the privacy setting of the container. Be aware that a new Container instance will always make a HEAD call, some level of caching or reuse is advised.

v0.8.4

v0.8.3

  • Keep connection alive; the header "Connection: Keep-Alive" is passed to the ObjectStore. This forces HttpClient to keep the connection open.
  • Account.list() and Container.list() hide implementation details; the ObjectStore returns a maximum of 10,000 containers or objects. When you called list(), you would get only this maxium and no more. To get it all, you had to use a Pagination map. With this release, you can call list() and it will return all containers/objects, hiding the max page size implementation detail.
  • Replace Mockito with JMockit; refactoring of the unit tests to make use of JMockit instead of Mockito, see also the blog post.
  • Public URL instead of internal URL; internal URL is intended for the hosting party, not for API usage. CloudVPS recommended to make use of the public URL returned by Keystone.

v0.8.2

v0.8.1

  • exists() does not log an error; everytime exists() is called, it used to log a 404 error. Since the call is about detecting, this has been disabled
  • custom host; not only mock, but also real mode benefits from having a custom host which you can use to prefix ObjectStore URLs
  • HttpClient socket timeout; if no custom HttpClient is supplied, JOSS can be instructed to instantiate a DefaultHttpClient with a custom socket timeout. In light of Release It!, useful to get more robust systems not clogging up on deadlocked HTTP clients
  • more clean ObjectDeleter; ObjectDeleter used to be a ScheduledExecutorService that always ran in the background. Now it only pops up when it is needed (for DeleteAt/DeleteAfter headers) and it shuts down when it is not
  • configurables in AccountConfig; easier to configure JOSS, with more variables now centralized in AccountConfig. The new variables are allowReauthenticate, allowCaching, mock allowObjectDeleter (see above), mock allowEveryone and mock onFileObjectStore
  • HEAD calls return SC_OK instead of SC_NO_CONTENT; Swift now returns SC_OK on all HEAD calls (account, container and object), instead of the SC_NO_CONTENT it had before. Mock mode has been aligned with this changed reality

v0.8.0

Theme: polishing for v1.0, adopting more requirements from project at 42

  • Encoding URLs; public URLs are encoded properly. This counts also for certain header values, such as X-Copy-From and X-Object-Manifest
  • Mock / Insert custom sequence of status codes; sometimes it is desirable that you can order JOSS to return a certain error. The system works somewhat similar to Mockito, so you can ask it to return a custom status depending on the nth call, a specific class, or both
  • Mock / Configurable delay; a mocked command call can now be ordered to Thread.sleep a set number of milliseconds to better simulate actual Swift conditions
  • Mock / Host; in mock mode, a custom host will be returned when a public URL is asked
  • Mock / major refactoring; the entire mechanism for mocking Swift has been refactored. It is now better aligned with the real implementation, ie more reuse of the same codebase. This issue was necessary to fix custom delays and custom sequences of status codes
  • Pagination Map query encoding; query parameters for pagination are now encoded as well
  • Updated credential classes; the latest Keystone implementation has a couple more fields, which JOSS now accepts as well

v0.7.0

Theme: plug'n'play for the convenience user and power for the power user

  • Paging lists of containers and objects; OpenStack paging can now be accessed either in the original form (using the last entry's name on the previous page as a marker), or with a page. Note that the list methods have change from listX to just list
  • Listing only containers and objects with a prefix; containers and objects can have their lists filtered on name prefixes
  • Public URL for mock implementation; a placeholder URL can be passed to the mock implementation. This URL will be prefixed to the container/object. In this way you can set up your own controllers that access content and still have it work in mock mode
  • Serialized output for containers; when containers and stored objects are listed, use is made of the JSON objects instead of the plain text lines
  • Reuse the header information from container/object listing for metadata; specific header fields passed by the Account/Container list operation are now stored and do not automatically lead to a refetch when queried (unless caching is turned off)
  • Minor changes/refactorings:
    • Check your calls to the ObjectStore; Account can let you know how many HTTP calls to the ObjectStore have been made so far
    • Logging critical information; various logging statements added (INFO, WARN and ERROR level), for configuration, critical components and exceptions

v0.6.1

Theme: First usage in the 42 project and feedback from the team

  • Large Object Creation; Segmented files and manifests can be uploaded. Files and byte array are automatically segmented if the segmentation size is exceeded. https://docs.openstack.org/swift/latest/overview_large_objects.html
  • Factories; factories that can be used in Spring to get a self-reauthenticating account singleton
  • OnFileObjectLoader; you can set up a folder with a layer of subfolders (ie, containers) and files (ie, objects). The ClientMock can be instructed to load this entire folder structure and convert it into an in-memory ObjectStore.
  • Thread safety; a number of routines have been made thread-safe so that the usage of a connection pool of accounts is not necessary, but a single Account can be instantiated in a Spring context
  • Other HTTP client; the default HTTP client now has a different connection manager (PoolingClientConnectionManager) than the default single-threaded one (BasicClientConnectionManager). You can still bring your own if this one is not good enough.
  • Caching on/off; JOSS lazy-reads the metadata on an instance and then never again. It is possible to turn off this caching mechanism, so that metadata is always loaded for every get call on account/container/object instance.
  • Entity reload; Account, Container and StoredObject can be forced to reload their metadata.
  • Minor changes/refactorings:
    • 100% Test coverage; all JOSS code is now covered by tests
    • Cyclic dependencies; eliminated all relevant cyclic dependencies from the code base
    • Public URL exposed by Account; Account now returns the public URL configured in the Object Store for the Account
    • Package refactoring; core Swift classes moved to model, original model classes moved to instructions and information

v0.5.0

Theme: Working down the list of unsupported Object Storage features from Future Features

  • X-Delete-After / X-Delete-At; with X-Delete-At, the ObjectStore can be instructed to remove an object at the specified date/time. With X-Delete-After, the ObjectStore is instructed to remove the object x seconds after the current date/time.
  • If-*Modified-Since headers; these headers instruct the ObjectStore to check against the last modification date. In the case of If-Modified-Since, the content will be returned if changes have taken place. In the case of If-Unmodified-Since, the content will be returned if changes have NOT taken place. See also, the Openstack API: https://github.com/openstack/swift/blob/master/api-ref/source/storage-object-services.inc#request
  • If-Match / If-None-Match; these headers instruct the ObjectStore to check against the passed value. In the case of If-None-Match, the content will only be returned if the value does not match (ie, content is modified). In the case of If-Match, the content will only be returned if the value matches (ie, content is unmodified).
  • Exception hierarchy; CommandException subclasses to prevent the need to compare exception content on catching. Every fail condition resulting from a HTTP status now translates to a corresponding exception.
  • Minor changes/refactorings:
    • Removed all IOExceptions from StoredObject interface
    • Ability to add Content-Type as part of the upload, not only afterwards
    • Removed dependency on commons.lang

v0.4.0

Theme: More functionality of Object Store in the client

  • download partial content; added Range that allow downloads of partials, see https://developer.openstack.org/api-ref/object-storage/#get-object-content-and-metadata
  • supply your own etag; added possibility to upload your own Etag, in case you use an inputstream but still require a server-side comparison of MD5 hashes
  • facilitate content sniffing; added option to change Content-Type of an object. Useful when you do not wish to rely on the default Object Store file extension matcher to determine Content-Type, eg when you have your own content-sniffing algorithm
  • better test coverage; 98%+ JUnit test coverage achieved

v0.3.0

Theme: CloudVPS feedback incorporated

  • added auto-reconnect after token expiration implemented
  • added auto-discovery of object-store Catalogs
  • added end point selection based on region preference
  • added tenant support

v0.2.0

Theme: Subjected to internal 42 review

  • converted InputStreamWrapper to InputStream subclass
  • return Containers and StoredObject as List instead of array[]
  • implements Callable interface for all AbstractCommand
  • use Closeable for AbstractCommand (added bonus: works for try-with-resources in Java 7 as well)
  • refactored client interface from Facade to ActiveRecord pattern

v0.1.0

First release with the basic authentication, account and object access