State-level VVSG 1.1 testing

Everyone Count’s Electronic Voting System “elect QuadAudit” passed state-level testing in compliance with the VVSG 1.1 standard. This makes elect QuadAudit the first system of its kind to pass the testing against VVSG 1.1 by a certified lab. elect QuadAudit is a complete electronic voting system that includes tablets for voter interaction, local servers for use within voting locations, and a management system for administration and tabulation. The system also complies with applicable accessibility standards.

Everyone Counts, Inc

Joined Everyone Counts, Inc as VP Product Engineering. This role covers all engineering aspects of Everyone Counts product portfolio, including the SaaS-based election platform for private and public elections.

Everyone Counts major products and services include Voter Registration, Electronic Poll Book, Online Voting, Voting System, Election Night Reporting and Election Administration.

Skylar LLC

After leaving Sony Electronics, Klaus established Skylar LLC. Skylar LLC is engaged in New Business Development, Technology Creation and Innovation through consulting and advisory roles. Skylar’s focus is on ideation, strategy definition, product viability verification, user experience, development execution and revenue generation. Currently Skylar is supporting start-ups in the San Diego area through the local accelerators Evonexus, Connect Springboard and Qualcomm Robotics Accelerator. Klaus is also advising Petwireless.com related to technology and business models.

Android TV

Android TV is the second and more refined approach by Google to bring Android to the TV (after the original Google TV). The Sony’s San Jose engineering team which belongs to Sony’s San Diego engineering group was on the front-line to work with Google and the other product partners. Products shipped in early 2015. See a video on YouTube about the CES feature demonstration.

Sony Android TV XBR 8505c

Sony Experience at Best Buy

Sony moved from their own retail stores to a store-in-store concept in partnership with Best Buy. More than 350 Best Buy locations within the United States have been selected to carry the “Sony Experience“, which for the first time featured Internet Access in the retail location. The San Diego team designed, developed and installed the UX, the on-location hardware and software. Network services include remote software update, content update to a local cache storage, system health monitoring and metric gathering. Customers can use a touchscreen to control the TV feature demonstration and product explanation, which would then show demonstration content (e.g. HDR, 4K, color space demos) on the appropriate TVs.

Sony Experience at Best Buy

Tennis Sensor

Sony brought a Tennis Sensor to the market, demonstrated at CES in January 2014. The unit attached to a tennis racket and measures among other things the impact speed of the ball and the area where the ball touched the racket. A companion application collects the data and provides useful statistics. The Verge published a review of the unit, that was first introduced in Japan, and shipped eventually in the US and elsewhere.

The San Diego team supported collaboration with Tennis Racket manufacturers, and created applications for the sensor, such as a video digest (i.e. based on the measured data and a video of a full training sequence, a tool would automatically create a collection of just forehand plays for further study).

Tennis Sensor measures performance

TV Production

The San Diego team was always very close to the manufacturing process: it was responsible for various device developments and production related to its own developments like 2nd Gen Google TV and Dash, and took care of the TV production in Tijuana. Sony sold the Tijuana factory in 2009, but Sony TVs are still produced there afterwards.  The Sony factory team joined Klaus’ Sony internal organization in 2013, mainly working on new model introduction, troubleshooting and logistics support.

Former Sony TV Manufacturing plant in Tijuana

Video Unlimited 4K Store

Sony launched the Video Unlimited 4K video distribution service along with the 4K media player FMP-X1. This world-first 4K consumer media player replaced the Sony 4K Experience equipment. The device had some videos preloaded on a hard drive, but customers could purchase new content from Sony’s Video Unlimited 4K service. The video store front end was developed by the San Diego engineering team.

Sony Video Unlimited 4K with the 4K media player FMP-X1

Xperia Info-Eye

Sony’s Xperia Z1 featured software and services that utilized the phone’s camera in new ways. The San Diego team specifically proposed, designed, and delivered the Info-Eye feature, which Engadget called “The glamor feature is Info-Eye, which gives you a visual search function on any captured landmarks or items like wine or books.”  Subsequent software updates added many new image recognition features, like “LinkedIn for Info-eye™” or Food Recognition.

Introduction of the Xperia Z1 with Info-Eye by Kaz Hirai

Usability Testing Group

Usability testing grows in importance for all Sony businesses. Some businesses do this for a long time, others are just beginning.  The San Diego team established a global Sony organization that supports all Sony business units to harmonize methodology and increase the global scope by offering a “one-stop-shop” for all things related to usability.

Usability Testing differs globally. The picture shows a third-party conducting a usability test for Sony in Copenhagen, managed by the San Diego team.

Watch Now

The Sony Watch Now application is an Electronic Program Guide (EPG) with social features, developed in San Diego. The idea behind the application is to provide a casual view of TV programming that is available right now, please see a video on YouTube about the app. Watch Now features metadata from Gracenote and allows to post to Facebook, Twitter and GetGlue. After integration with Sony’s newer “TVSideView” application, the original Watch Now was discontinued.

Watch Now included the ability to control a TV through IR or IP, so that customers could tune to the selected program from the app.

Watch Now Screenshot

Sony 4K Experience

Sony introduced the worldwide first 4K content distribution system with its first 84″ 4K TV. The complete system comprised a 84″ 4K TV, along with a PC based 4K content player, a tablet to control the 4K content player and a backend service – the content delivery service with feature films was a world-wide first, and a Sony differentiation for more than a year. The tablet application, the 4K server and the content processing pipeline have been developed and operated by the San Diego engineering team. CNet has a video that shows the tablet application in action.

The complete system was announced at CES 2012 by Phil Molyneux (COO and President of Sony Electronics), and developed from scratch afterwards.

CNet Video Capture showing Tablet App

Fitness Portal on TV

The San Diego team proposed, developed and eventually shipped a Fitness Portal for Sony BDP players. This portal allowed customers to access video content from various providers related to fitness such as Yoga or exercise instructions – partly behind a pay-wall – and see stats from connected wearable devices like Fitbit or Withings. The idea behind the portal was to create a separate destination for fitness content on the TV and bring in personalized data, and by doing so providing better access to such material as the apps are taken out of the “sea of icons” from the general application launcher.

Fitness Services on the left, Fitbit/Withings not connected (no public image with this feature found)

2nd Gen. Google TV

The Sony Google TV Box NSZ-GS7 was to a large degree developed by the Sony San Diego engineering team, including industrial design, hardware and software development, as well as overall system integration including day-to-day work with SoC vendor.

Compared to the first generation, this device featured a greatly enhanced remote control, which received several awards by itself. The remote control was subject of intense user testing as a result of the rather mixed feedback from the first Google TV generation.

Sony Google TV Box NSZ-GS7

Dreamhouse

The San Diego team established a “Dream House” in San Diego with all the good Sony equipment, full home network and home automation installation, and a user experience / user testing facility. Sony’s social video network SGNL covered it in 2011. There is another actually quite funny advertisement movie shot at the Dream House available at YouTube. Other than user testing, the main focus of the Dreamhouse was to bring the Sony equipment together and link with home automation, specifically using Control4 technology.

Sony Dreamhouse YouTube Video

Socialize on BluRay Player

The 2011 Blu-Ray Players included a feature that was invented, implemented and operated by the San Diego team: it was possible to watch a video and create a simple message for Facebook or Twitter with the remote control. The message was basically a link to the video that was watched, and a selectable pre-defined text, like “is watching” or “enjoyed”. Below is a picture of the resulting Facebook page. This might have been the first time of a direct integration of social media in a BDP user interface.

Facebook posting as a result of using the Socalize feature

Several business opportunities are related to this feature. First, through Facebook Insights, a rich amount of data related to the users of the feature was available. Secondly, the posting on Facebook or Twitter shows the fact that this post was done through a Sony BDP and there was an easy way to “like” that fact. And finally, the link to the video can be monetized as referral, if that a click on the link results in a purchase of related content.

Advertisement on TV/BDP

Starting with the 2011 TV models, Sony XBR Televisions featured a new user interface designed by the San Diego team. This UI also included a “Recommended” menu item, which in fact was used for advertisement and other messaging to generate after-sales revenue. It was the first time to include advertisement on the main TV user interface, probably industry-wide.

The San Diego advertisement approach was basically about selling the ad-space to advertising agencies, so that Sony would not deal with advertisement aggregation. The system was later expanded to Sony Blu-Ray players.

Recommendations in the TV UI

Developer.sony.com

The San Diego team launched in 2011 the third Sony party developer support site “sonydeveloper.com” as a local initiative – Klaus personally acquired the initial domain name to accelerate the deployment at the time. The purpose was to begin a dialog and attract developers to create added value to Sony products. The site supported a San Diego proprietary SDK that was intended to run on some TV platforms, information about development of Yahoo widgets and the developer support for the Dash device. Eventually the site became http://developer.sony.com and integrated with Sony Mobile’s third-party development portal. The site is still operated in parts based on the original infrastructure. It is renamed “Developer World” and now the main destination for all Sony developer relations (other than PlayStation).

Sony Developer World Screenshot

1st Gen. Google TV

Sony rolled out the first generation of Google TV in October 2010, coming out in form of TVs and a Blu-Ray player. The San Diego team was involved in the partnership with Google through a San Jose-based team, and through application development for Google TV. The applications included a social media application that allowed to integrate Twitter and Facebook feeds into one stream, focusing on video and picture content. The San Diego team expanded its contribution to the Google TV product line with the second generation.

NSX-24GT1, the 24″ HDTV with Google TV

Dash

The Sony HID-C10, also known as “Dash”, was developed and brought to the market by the San Diego team, launching in April 2010. The Dash is a “personal Internet Viewer”, based on some of the software introduced by the Chumby device. But Dash integrated also other services, including BRAVIA Internet Video, which provided Netflix, Amazon Instant Video and YouTube. A full WWW browser was added later.

The Dash Personal Internet Viewer

Yahoo Widget on BRAVIA

The 2009 Sony TV included the Yahoo Widgets. Widgets are small applications written in Java Script that could overlay the TV screen. There was a dedicated button on the remote control to launch the widgets. The San Diego team worked with Yahoo to bring the service to the TVs and expanded it’s functionality. For example, some of the Sony original TV applications have been using JavaScript as well, allowing to more easily update the application and mange the system memory dynamically.

Weather Widget on the TV Screen

Bravia Internet Video

The original BRAVIA Internet Video Link device DMX-NV1 shipped in July 2007, just before Klaus joined the San Diego team. The device connected to specific BRAVIA TVs and provided for the first time in the industry streaming internet video to consumer TV, integrated with the main user interface, and therefore creating the world’s first “Smart TV”. Three IPTV services have been supported at the time, including AOL and Yahoo. The development in San Diego included the service backend.

BRAVIA Internet Video Link (BIVL) attached to a compatible 2007 TV

IPTV Service Operation

With the start of the BRAVIA Internet Video, the San Diego team was also in charge of operating the IPTV backend. The backend was actually not serving video, it just managed the service appearance on the TV and later BDP. The backend provided metadata about customer usage of the service, and allowed Sony to enable and disable services for specific regions, devices and schedules.

Service List on 2011 BDP in the Cross Media User Interface

PlayStation F@H

The Folding at Home application on PS3 was developed by Sony Computer Entertainment, managed by Klaus in Foster City. It was part of the PS3 firmware from 2007 to 2012. Once released, it did not take long to reach more than one petaflop sustained distributed computing performance, a Guinness record – see also the Guinness site. The application was very popular, and there are people who bought a PS3 exclusively for the purpose of running Folding at Home.

PS3 Folding at Home Screenshot

PS3 Launch

The Playstation team in Foster City provided several software components for the PlayStation 3 platform itself, and helped third parties with the launch titles. This included specifically the OpenGL library that was utilized by many game titles.

PlayStation 3 Launch Console

Startup: TwonkyVision

Klaus was original co-founder of the TwonkyVision GmbH in Berlin Germany in 2000. The company developed a variety of media software, including an MHEG Engine and the well-known Twonky Media DLNA Server. TwonkyVision was financed privately and grew organically. The company was successfully acquired in 2006 by PacketVideo in San Diego. Klaus’ work for TwonkyVision was done in parallel and with approval by Sony.

The Twonky Media Server was as market leader deployed on many Network Attached Storage devices (e.g. Western Digital), but the PC version was quite popular as well. The main differentiation from others was a high compatibility rate and small memory footprint.

Twonky Media Server Configuration on Windows

Twonky Media Content Player

VAIO Media Server

Sony VAIO computers shipped starting 2004 with a VAIO Media Server developed by Klaus’ San Jose team. The server was based on UPnP (Universal Plug and Play) and could steam music to other devices in the home.

Universal Plug and Play Logo

Flash Player on CLIE

Sony’s CLIE handheld devices running Palm OS supported after 2003 Flash file playback. The player was developed by Klaus’ team in San Jose, including all conformance testing. One of the prominent CLIE’s with Flash support was the CLIE PEG-UX50.

CLIE PEG-UX50

OSGi Gateway

Klaus was involved in the Open Services Gateway Initiative from the very beginning as Sony representative, contributor and later developer.

OSGi defines a Java based dynamic runtime environment for services intended to run on small footprint devices. Service (“bundles”) could be dynamically loaded and managed depending on application needs. This environment was specifically suitable for Set Top Boxes and Home Gateways. While there was Sony internal development, OSGi compliant products never shipped.

OSGi Overview by Peter Egli

XHTML Contributor

From Wikipedia: Extensible Hypertext Markup Language (XHTML) is a family of XML markup languages that mirror or extend versions of the widely used Hypertext Markup Language (HTML), the language in which Web pages are formulated.

Klaus participated as contributor in the World Wide Web Consortium’s XHTML working group. The standard version 1.0 released in early 2000. As an application of XML, XHTML was intended to provide a cleaner and more verifiable version of HTML.

SMIL Contributor

From Wikipedia: Synchronized Multimedia Integration Language (SMIL) is a World Wide Web Consortium recommended Extensible Markup Language (XML) markup language to describe multimedia presentations.

Klaus also participated in the definition of SMIL, the recommendation 1.0 was released by W3C in April 1998.

MHEG-5 Chairman

From Wikipedia: MHEG-5 is a licence-free and public standard for interactive TV middleware that is used both to send and receive interactive TV signals. It allows a wide range of TV-centric interactive services to be deployed. It is used by Freeview and Freesat in the UK, Freeview in New Zealand, TVB in Hong Kong, Freeview in Australia, Saorview in Ireland and has been specified in South Africa.

Klaus served as Chairman of the ISO/IEC SC29WG12 group that developed and defined the standard for a few years, until the standard release 1997. Klaus was also the official representative of the German National Standardization Institute DIN.