PiCloud Platform
PiCloud is a cloud-computing platform that enables you to leverage thousands of cores of computational power from Amazon Web Services without having to manage, maintain, or configure virtual servers.
You interface with PiCloud using our cloud library, command-line interface (CLI),
and web dashboard. The cloud library is written in Python, and can offload any
Python function in as few as two lines of code. However, PiCloud supports running programs
created in any language.
PiCloud removes all the plumbing necessary to get your functions and programs running on a high-performance compute farm. As you give us more computation, our cluster automatically scales to meet the demands of your workload. We take responsibility for the security of your data, and ensure high availability and reliability regardless of underlying server failures.
PiCloud gives every scientist, developer, and engineer in the world access to clusters of machines that 10 years ago were available only to the largest companies and national laboratories. Know that when using the platform, you are supporting a product with the goal to democratize computing power around the globe.
Service Highlights
Easy to Use
We offer a superior cloud experience encapsulated entirely within your programming language. With as few as 2 lines of code, PiCloud will leverage the computational power of a cluster of servers for you. We also offer a streamlined web dashboard for customization and monitoring on-the-go.
Elastic
PiCloud is always ready to accept and process your function. Behind the scenes, we are automatically scaling our service to match your computational needs from no load to peak usage. Code in full confidence that your service will scale.
As you submit functions to PiCloud, they are added to our queue of functions waiting to be processed. To handle the computational load from our users, we automatically scale our cluster by provisioning more servers. In this way, we are able to handle your highly variable workload without any effort on your part. If you require guarantees on how quickly your functions begin executing on our cluster, see our real-time feature. To learn more, please see our technical overview.
Robust
PiCloud brings a highly robust computing environment to the cloud. Amazon and other cloud providers explicitly state that their servers can and do go down without notice due to hardware and network failure, and that it is your responsibility to build robustness into your systems. PiCloud has built redundancy and reliability into every corner of its service. The PiCloud Service Level Agreement commitment is 99.9% availability.
Secure
PiCloud employs industry-standard security techniques. All communication with PiCloud is encrypted using the Secure Sockets Layer (SSL) protocol. Your code and data on our cluster is protected by multiple layers of security including POSIX permissions, AppArmor, and Kerberos. In addition, PiCloud deploys the latest security patches to give you maximum protection. PiCloud also offers a dedicated cores option to guarantee that your computation runs in isolated VMs.
Inexpensive
PiCloud passes the financial benefits of its multi-tenant environment on to you. On Amazon, you'll be paying for servers by the hour whether or not you use them. PiCloud only charges you for the exact amount of computation time you use, giving you the ability to parallelize your computation cheaply. Never again will idling servers drive up your infrastructure costs. Heavy users can purchase compute resources in bulk for further discounts.
Features
Automated Deployment
With PiCloud, you'll never see a server again. Your program's execution state is seamlessly migrated to the cloud
without any user intervention. Dependencies, such as custom Python packages, are automatically extracted by the
cloud library and distributed across PiCloud's machines. All other types of packages (binaries,
modules) can easily be deployed with our easy-to-use
environment creation tool. The days of repetitively templating, versioning, provisioning, and deploying
are over.
Tunable Performance
PiCloud enables you to run your functions with a variety of different compute resources. For simple tasks or testing, a c1 core with its limited compute power and memory may be sufficient. CPU bound applications will benefit greatly from the c2 cores, which have the power of 2.5 compute units. m1 cores make it possible to process large data sets in memory. Switching between cores takes a second, allowing you to tune and optimize your performance easily.
| Type | Compute Resources |
|---|---|
| c1 - default |
|
| c2 |
|
| m1 |
|
| s1 |
|
Real-Time Computing
For computation with strict processing time requirements, PiCloud offers real-time cores. Real-time cores are reserved on our cluster solely for you, and are ready at a moment's notice. They supplement the computing power available with our standard service. Functions you submit will be executed immediately if you have real-time cores available. As your functions complete, they free up real-time cores for your other functions that are still waiting in the queue. Therefore, the number of real-time cores you reserve determines the number of your functions that are guaranteed to be executing in parallel at any given point in time. You can reserve real-time cores by the hour. To learn more, please see our post on how to get more parallel computing power.
Fully Customizable
With our environments feature, your computation can use any program or library written in any language. This includes applications written in Java, C, C++, R, Matlab, and more. To learn more, please see our post introducing environments.
Performance Analytics
As a language-integrated platform, PiCloud analyzes your running code and reports its resource usage on a per function basis. The information is accessible via PiCloud's web interface, allowing you to quickly see inefficiencies.
Monitoring and Error Handling
PiCloud consolidates your computation history into one simple interface. You can examine the innermost details of any function including critical information such as standard output, standard error, profile, and even exception tracebacks.
Scientific Computing Ready
PiCloud has partnered with Enthought, Inc., the market-leader in scientific computing with Python, to bring rigorous data analysis and visualization tools to the cloud. PiCloud supports and has optimized the use of packages included in the Enthought Python Distribution (EPD). A full list of libraries is available here.
Files and Shared Memory
PiCloud instantly works with any existing data store you are currently using, whether it be a database or your own Amazon S3 bucket. We do not lock in data. For those without an existing solution, PiCloud offers you the ability to store and retrieve data (files or memory objects) in a bucket, or on a distributed file system.
Crons
Using crons, you can schedule functions to run periodically at specific times and dates. Crons can be triggered as often as every minute, and there’s no limit to the number of functions you can register as crons.
Algorithm Publishing
If you publish a Python function to PiCloud, you can invoke it from any language or device that can issue web requests. Invocation is done by accessing the published function's associated URL, where arguments are passed in as POST parameters. To learn more, see our post on publishing functions.
Clouds
Amazon Web Services
PiCloud's primary deployment is on top of Amazon Web Services (AWS) in the US-EAST Region. If you are already using AWS in the same region, your latency will be minimal, bandwidth will be high, and you will not have to pay for data transfer costs.
Private Datacenters
PiCloud offers an installation for private datacenters. We can deploy our systems on your clusters ranging from ten machines to thousands, running VMWare, Xen, or bare metal stacks. To learn more, please contact us.
Pricing
Please see our pricing page.
System Details
Compute Units Defined
A compute unit is defined by Amazon to be equivalent to a 1.0-1.2 GHz 2007 Opteron or 2007 Xeon processor. We use Amazon's standard since our systems are currently built on their infrastructure.
Bandwidth and Latency
If you are already on Amazon Web Services in the US-EAST Region, communication latency is minimal and is equivalent to the latencies between any two Amazon EC2 hosts. Likewise, PiCloud's bandwidth is equivalent to other hosts on Amazon Web Services. For those elsewhere, it is important to measure the latency and bandwidth between you and PiCloud to properly gauge PiCloud's effectiveness. You can measure your latency by pinging picloud.com.
Transfer of Code and Data
PiCloud's programming library, cloud, automatically transfers the necessary source code, byte code, execution state,
and data to run your function on our cluster. Our library is careful to only transfer the code and data it needs,
and all information is encrypted using SSL before transmission. Caching is present in both the client library and
our cluster so that future requests do not transfer redundant information. Our library can also be configured
to log what information is being transferred for your inspection.
Performance Optimizations
PiCloud has built a proprietary distributive system specifically to maximize performance and scalability on elastic farms of virtual machines. The following is only a subset:
- We've benchmarked every corner of AWS, and are giving you the best combination of speed and price that we've discovered.
- We've eliminated the overhead of spawning new Python Interpreter's, importing standard Python libraries, and even re-importing your custom modules for minimum delay.
- Our systems are custom compiled and fine tuned for the machines and architectures available to us on the cloud. Your functions immediately take advantage of our optimizations.
- Our predictive systems handle provisioning and scaling taking into account all the nuances of the underlying infrastructure.
Documentation
Please see our Python cloud library documentation.