diff --git a/content/news/258-dev-feb20.md b/content/news/258-dev-feb20.md index 9c62d3b5..ee06fac0 100644 --- a/content/news/258-dev-feb20.md +++ b/content/news/258-dev-feb20.md @@ -12,7 +12,7 @@ To read previous updates, please visit our [Development and Community Update arc If you want to see a condensed view of what we've completed recently and what’s planned for LBRY, check out our [Roadmap](https://lbry.com/roadmap). We’ve recently updated it with our 2020 priorities, and you can watch our [video overview on lbry.tv](https://lbry.tv/@lbry:3f/LBRY-2020-Roadmap-Release:6). -Let’s start with some GitHub stats across all our repos since our last update about 51 days ago. 45 repos were updated. 770 issues were created, 100 were closed. 21 PRs were merged. 13 GitHub users, outside of LBRY, made code contributions: [kodxana](https://github.com/kodxana), [andbeletsky](https://github.com/andbeletsky), [Meantub](https://github.com/Meantub), [TigerxWood](https://github.com/TigerxWood), [Yamboy1](https://github.com/Yamboy1), [dalhill](https://github.com/dalhill), [eggplantbren](https://github.com/eggplantbren), [er013](https://github.com/er013), [joebass85](https://github.com/joebass85), [michaeltintiuc](https://github.com/michaeltintiuc), [mkroman](https://github.com/mkroman), [nestordominguez](https://github.com/nestordominguez), [osilkin98](https://github.com/osilkin98), [tuxfoo](https://github.com/tuxfoo), and [ykris45](https://github.com/ykris45). +Let’s start with some GitHub stats across all our repos since our last update about 51 days ago. 45 repos were updated. 770 issues were created and 100 were closed. 21 pull requests were merged. 13 GitHub users outside of LBRY made code contributions: [kodxana](https://github.com/kodxana), [andbeletsky](https://github.com/andbeletsky), [Meantub](https://github.com/Meantub), [TigerxWood](https://github.com/TigerxWood), [Yamboy1](https://github.com/Yamboy1), [dalhill](https://github.com/dalhill), [eggplantbren](https://github.com/eggplantbren), [er013](https://github.com/er013), [joebass85](https://github.com/joebass85), [michaeltintiuc](https://github.com/michaeltintiuc), [mkroman](https://github.com/mkroman), [nestordominguez](https://github.com/nestordominguez), [osilkin98](https://github.com/osilkin98), [tuxfoo](https://github.com/tuxfoo), and [ykris45](https://github.com/ykris45). Thanks to everyone who took time from their busy days to help LBRY out! @@ -28,42 +28,42 @@ Thanks to everyone who took time from their busy days to help LBRY out! * [2019 review and new 2020 roadmap](#roadmap) ## Desktop app {#summary-desktop} -Since our last development update, the app team shipped two named releases: [0.39.0](https://github.com/lbryio/lbry-desktop/releases/tag/v0.39.0), under codename [Joule](https://lbry.com/news/joule), and [0.41.0](https://github.com/lbryio/lbry-desktop/releases/tag/v0.41.0), under codename [Kelvin](https://lbry.com/news/kelvin). Both continue to improve user experience through homepage improvements and a new repost feature. Version 0.40 was skipped due to a bad build, and [0.42.0](https://github.com/lbryio/lbry-desktop/releases/tag/v0.42.0) finished up the repost work started in Kelvin by allowing them to be created in the directly in the app. +Since our last development update, the app team shipped two named releases: [0.39.0](https://github.com/lbryio/lbry-desktop/releases/tag/v0.39.0), under codename [Joule](https://lbry.com/news/joule), and [0.41.0](https://github.com/lbryio/lbry-desktop/releases/tag/v0.41.0), under codename [Kelvin](https://lbry.com/news/kelvin). Both continue to improve user experience through homepage improvements and a new repost feature. Version number 0.40 was skipped due to a bad build, and [0.42.0](https://github.com/lbryio/lbry-desktop/releases/tag/v0.42.0) finished up the repost work started in Kelvin by allowing them to be created in the directly in the app. ### Joule improvements - homepage, invites, and more Joule brought a huge revamp to the homepage experience by moving away from the default trending + tags view, to a more dynamic and larger tile listing of your preferred content. -We want to make the experience more about the content you’re already interested in versus trying to force you into a trending first discovery as in previous versions. The other main feature in this release was invites. We re-worked this area allowing users to share special invite URLs featuring their channels instead of a random invite code. Inviting friends and fans of your channel has never been easier! +We want to make the experience more about the content you’re already interested in, as opposed to trying to force you into trending-first discovery as in previous versions. The other main feature in this release was invites. We re-worked this area allowing users to share special invite URLs featuring their channels instead of a random invite code. Inviting friends and fans of your channel has never been easier! -A bit more on the UI changes: front and center is your Following feed, which includes 2 rows of the latest content from your subscriptions. Below this you are presented with trending content for some of your tags, then trending across LBRY, and finally content from the @lbry channel to help you stay up to date with the latest news and happens on our platform. Each of these sections can be drilled into via the hyperlink to a more detailed listing. We moved from dropdowns to more user-friendly buttons to sort by top/trending/new. The sidebar is also a bit more dynamic and shows tags/followers based on your view. +A bit more on the UI changes: front and center is your Following feed, which includes two rows of the latest content from your subscriptions. Below this you are presented with trending content for some of your tags, then trending across LBRY, and finally content from the @lbry channel to help you stay up to date with the latest news and happens on our platform. Each of these sections can be clicked into via the hyperlink to a more detailed listing. We moved from dropdowns to more user-friendly buttons to sort by top/trending/new. The sidebar is also a bit more dynamic and shows your followed tags or channels, depending on your choice of view. ![Joule](https://spee.ch/@lbrynews:0/lbry-joule.gif) ### Kelvin features - reposts, top claims, autoplay, and more In Kelvin, we added the ability to display reposts. This is a new feature created for LBRY and is similar to what you’ve seen on platforms like Twitter where you can repost someone else’s content to your own feed. -We are hoping it will be used to help in the discovery process as it allows creators to share content from other creators. It also enables a cool new type of channel creating a curation of content. We call these repost only channels (check out [lbry://@Top-LBRY-YouTubers](https://open.lbry.com/@Top-LBRY-YouTubers)). Another cool use case is the ability to repost your own or others’ content under a different URL (i.e. when you go to lbry://lbryrocks it redirects to lbry://jiggytom). +We are hoping it will help in the discovery process as it allows creators to share content from other creators. It also enables a cool new type of channel, dedicated to curating content. We call these repost only-channels (check out [lbry://@Top-LBRY-YouTubers](https://lbry.tv/@Top-LBRY-YouTubers:d) for an example). Another cool use case is the ability to repost your own or others’ content under a different URL (i.e. when you go to lbry://lbryrocks it redirects to lbry://jiggytom). -On LBRY, reposts work by creating a new claim referencing the content you are reposting, and signs it with your channel. Attribution still goes to the original claim holder, i.e. tips go to them, but your channel can still get tipped for having a well curated feed or helping share content from other creators you enjoy. Version 0.42 introduced the ability to make reposts directly from file pages. By default, it keeps the same claim name (and bid), but you can also change the claim name (similar to the jiggytom example above) to anything you want. Sometimes it requires you to do so as the app enforces a single claim name per wallet limit currently. +On LBRY, reposts work by creating a new claim, which references the content you are reposting, and signs it with your channel. Attribution still goes to the original claim holder, i.e. tips go to them, but your channel can still get tipped for having a well curated feed or helping share content from other creators you enjoy. Version 0.42 introduced the ability to make reposts directly from file pages. By default, it keeps the same claim name as the original post, and specifies a low bid. However, you can also change the claim name and bid to anything you want (e.g., with the jiggytom example above, the claim name was changed, and you might want the bid to be high enough to obtain the vanity name). Sometimes it requires you to do so as the app currently doesn't allow a wallet to have two claims with the same name at present. -We also added a top claims page when typing any URL in the search bar, see [lbry://elon-musk](https://lbry.tv/$/top?name=elon-musk). This page also allows you to directly support this content to help it appear higher in the search and trending. Finally, other new features include a download button on claim hover, new icons that show the file type, and the ability to edit/delete your own comments. +We also added a top claims page when typing any URL in the search bar, see [lbry://elon-musk](https://lbry.tv/$/top?name=elon-musk). This page also allows you to directly support this content to help it appear higher in the search and trending. Finally, other new features include a download button on claim hover, new icons that show the file type, and the ability to edit and delete your own comments. ![Kelvin](https://spee.ch/5/lbry-kelvin.gif) ### Other features, tweaks, and bug fixes Other new features and changes in Joule/Kelvin, and since our last update include: -- Autoplay next improvements - a countdown/cancel button, prevent playing channels/blocked content, remove mature recommendations on non-mature content +- Autoplay next improvements: a countdown/cancel button, prevent playing channels/blocked content, remove mature recommendations on non-mature content - More languages: Danish, Romanian, and Urdu. - Improved display and handling of text content including markdown. - AppImage support on Linux, including seamless auto upgrades! - Authentication data now stored in a cookie rather than keyring (including a migration). - Ability to unfollow deleted channels. -- Fix followed channel syncing between devices. -- Thumbnail of content auto generated on lbry.tv when no thumbnail is present. -- Improved embed links - now support Twitter video, thumbnails, and better autoplay support (off by default). +- Fix followed-channel syncing between devices. +- Thumbnail of content auto-generated on lbry.tv when no thumbnail is present. +- Improved embed links - with support Twitter video, thumbnails, and better autoplay support (off by default). ### Privacy at LBRY -If you’ve checked out the @lbryio channel recently, you may have seen our [latest video explaining privacy changes coming to LBRY](https://lbry.tv/@lbry:3f/privacy-update:f). On the desktop side, it will now be clearer which types of data is tracked and by whom, and users will have control over their settings. Both new and existing users will be prompted with the above welcome screen on startup. Users who choose to sign into lbry.tv on desktop will have to at least share data with LBRY, to be eligible to earn rewards. Telemetry and other analytics data through Google Analytics will be opt in. +If you’ve checked out the @lbry channel recently, you may have seen our [latest video explaining privacy changes coming to LBRY](https://lbry.tv/@lbry:3f/privacy-update:f). On the desktop side, it will now be clearer which types of data is tracked and by whom, and users will have control over their settings. Both new and existing users will be prompted with the above welcome screen on startup. Users who choose to sign into lbry.tv on desktop will have to at least share data with LBRY to be eligible to earn rewards. Telemetry and other analytics data through Google Analytics will be opt-in. ![privacy](https://spee.ch/4/privacy-lbry.jpeg) @@ -96,10 +96,10 @@ The main features and bug fixes include: - Torba, the wallet, is now part of the main lbry-sdk codebase. - Ability to update comments (editing). - Wallet sync speed improvements through batching. -- UPNP improvements to support more routers. +- UPnP improvements to support more routers. - Add ability to pass blocking and filtering channels to wallet server configurations. - Add Prometheus metrics for better monitoring. -- Add additional trending algorithm called AR which is more responsive by recalculating every 10 blocks (Thanks EF!!). +- Add additional trending algorithm called AR which is more responsive by recalculating every 10 blocks (Thanks @EF). - Resolve flag to claim/stream/channel list (to get canonical urls and other trending metadata). ![sync](https://spee.ch/b/headers-sync.png) @@ -109,13 +109,13 @@ The latest SDK includes the ability to transcode content during the publish proc **--validate_file** - validate that the video container and encodings match common web browser support or that optimization succeeds if specified. **--optimize_file** - transcode the video & audio if necessary to ensure common web browser support. -The first will simply tell you whether or not the file is within spec for typical streaming scenarios at LBRY (i.e. it will check the bitrate, that it’s web optimized - faststart, and the correct encoding) and won’t actually transcode. The 2nd flag is the one that does the transcoding to proper spec (h264/AAC, faststart, streamable bitrate). A new file is created along side the original in the same folder, which is then used to upload to LBRY. Currently this file must be manually deleted. +The first will simply tell you whether or not the file is within spec for typical streaming scenarios at LBRY (i.e. it will check the bitrate, that it’s web optimized - faststart, and has the correct encoding) and won’t actually transcode. The 2nd flag is the one that does the transcoding to the most compatible spec (h264/AAC, faststart, streamable bitrate). A new file is created along side the original in the same folder, which is then used to upload to LBRY. Currently this file must be manually deleted. ![validate](https://spee.ch/0/sdk-validate.png) ### Claim search indexes added -We recently revamped our [wallet server indexes](https://github.com/lbryio/lbry-sdk/pull/2811) to include additional sorting and filtering options so that the Desktop app can give users advanced discovery options. This includes duration, claim type, stream type, fee amount, and channels. We also cleaned up some old indexes for trending global that were not being used. Currently there is a limitation with SQLite where only 1 index can be used at a time and we hope we can take more advantage of multiple indexes when moving to a POSTGRES database. +We recently revamped our [wallet server indexes](https://github.com/lbryio/lbry-sdk/pull/2811) to include additional sorting and filtering options so that the Desktop app can give users advanced discovery options. This includes duration, claim type, stream type, fee amount, and channels. We also cleaned up some old indexes for trending global that were not being used. Currently there is a limitation with SQLite where only one index can be used at a time and we hope we can take more advantage of multiple indexes when we move to a POSTGRES database. ### Faster sync and performance improvements The SDK team has also been working closely with the blockchain team on sharing/consuming data directly without having to sync. Part of this work will also allow us to make the client/server work directly together without having to make additional calls to sync address data, transaction history, and to make claim search/resolve calls. This is going to speed things up considerably for the lbry.tv use case. @@ -127,27 +127,27 @@ We’ve made additional progress on our ability to download Torrent files direct After the silent lbry.tv launch, we officially launched lbry.tv at the end of January, and have been heavily promoting it throughout February. We’ve had tons of new users sign up for the LBRY app experience on the web and we are thrilled with the growth and attention we are seeing! -With this large and quick growth, we’ve also experienced technical and resource growing pains. Although we’ve added additional streaming servers in the US, they are sometimes still overloaded causing buffering issues. Our plan to to add additional servers here and around the world in the next couple of weeks - lbry.tv will be able to figure out which server is the best for a particular user. +With this large and quick growth, we’ve also experienced technical and resource growing pains. Although we’ve added additional streaming servers in the US, they are sometimes still overloaded, causing buffering issues. Our plan to to add additional servers here and around the world in the next couple of weeks - lbry.tv will be able to figure out which server is the best for a particular user. On the SDK side, we’ve also seen large loads due to many wallets being loaded and tons of wallet related activity. Some of the calls are not optimized yet, which may bog down the experience for some users. This led us to scale up with additional SDK servers, and we are currently moving to a cluster solution to support more traffic across multiple server instances. We are also re-working some of the inner designs of how the SDKs interact with wallet servers to provide the most optimized environment for users (there’s really no need to have these databases on separate servers like the typical LBRY app > wallet server connection we have today). We are excited to see how performance improves once we get the larger server and SDK improvements in place. Thank you for bearing with us during some of the downtimes and slowness during this large growth period. We promise a much smoother experience is on the horizon. ### Monitoring data and metrics -During the last couple of months, we’ve beefed up our system monitoring and metrics tools to get a better understanding of the lbry.tv environment and the pieces that come into play. We’re integrating Prometheus into more of the various layers, and using a tool called Grafana to put together some fantastics metrics and monitoring tools so we can understand what’s happening at a glance. +During the last couple of months, we’ve beefed up our system monitoring and metrics tools to get a better understanding of the lbry.tv environment and the pieces that come into play. We’re integrating Prometheus into more of the various layers, and using a tool called Grafana to put together some fantastic metrics and monitoring tools so we can understand what’s happening at a glance. ![metrics](https://spee.ch/9/metrics-lbry.jpeg) ### Dedicated streaming servers -As LBRY becomes increasingly more popular, we have to ensure the network remains stable and fast when it comes to content delivery. For this reason, we provisioned multiple servers with absurdly high capacity (almost 50TB of space) to better cache and quickly deliver content straight to our users. If you're interested in running your own content storage you should stay tuned as we will likely release instructions on how to better help the network in mass. +As LBRY becomes increasingly more popular, we have to ensure the network remains stable and fast when it comes to content delivery. For this reason, we provisioned multiple servers with absurdly high capacity (almost 50TB of space) to better cache and quickly deliver content straight to our users. If you're interested in running your own content storage you should stay tuned as we will likely release instructions on how to better help the network en masse. -One key component that we detached and deployed as standalone service is the streaming server itself, which used to be part of lbry.tv. It now has its [own repository](https://github.com/lbryio/lbrytv-player) and runs on several dedicated servers. The improvement was very noticeable: the website became snappier, loaded faster and most importantly, videos played more reliably! +One key component that we detached and deployed as a standalone service is the streaming server itself, which used to be part of lbry.tv. It now has its [own repository](https://github.com/lbryio/lbrytv-player) and runs on several dedicated servers. The improvement was very noticeable: the website became snappier, loaded faster and most importantly, videos played more reliably! We intend to continue working on scaling lbry.tv better and reaching regions all around the world providing an experience at least comparable to the one North Americans are having now. ### Invite page -With our invites improvements, there is not a dedicated landing page for content creator’s channels when they share their invite links. Once a user hits an invite URL, they are automatically subscribed to the channel, the invite is credited, and new users can proceed to create a lbry.tv account. +With our invites improvements, there is now a dedicated landing page for content creator’s channels when they share their invite links. Once a user hits an invite URL, they are automatically subscribed to the channel, the invite is credited, and new users can proceed to create a lbry.tv account. ![invite](https://spee.ch/6/invite-page.jpg) @@ -192,17 +192,17 @@ Next steps for Android include the similar first run changes as Desktop where we ![follow](https://spee.ch/d/Follow-android.jpg) ### F-Droid -As part of our privacy initiative for DTF February, we’ve begun making the necessary changes to our build process and removing analytics/Firebase so that we can be officially part of a the F-Droid repository. Both of these were requirements in order to be considered and require a standalone SDK build without buildozer and a refactor of our code to easily remove GA. Since Firebase will also be removed, these versions will not have notification capabilities until we can replace it with a non-Google alternative. +As part of our privacy initiative for DTF February, we’ve begun making the necessary changes to our build process and removing analytics/Firebase so that we can be officially part of the F-Droid repository. Both of these were requirements in order to be considered and require a standalone SDK build without buildozer and a refactor of our code to easily remove GA. Since Firebase will also be removed, these versions will not have notification capabilities until we can replace it with a non-Google alternative. ## Blockchain {#blockchain} In the last couple of months, the blockchain team has been working on two main tasks - assisting the SDK team with a single claim database mechanism and bringing in Bitcoin version 19 upstream changes. The claim database redesign will allow the LBRY SDK to directly access claim data without having the wallet servers needing to sync from LBRYcrd, which is currently a slow and tedious process. -The Bitcoin version 19 changes will bring additional stability improvements to our codebase, but will force us to revamp how our tipbots work since the accounts mechanism is removed in this version. The team has also been assisting with providing the video transcoding capabilities in the SDK and debugging slowness of wallet related calls. +The Bitcoin version 19 changes will bring additional stability improvements to our codebase, but will force us to revamp how our tipbots work since the accounts mechanism is removed in this version. The team has also been assisting with providing the video transcoding capabilities in the SDK and debugging slowness of wallet related calls. ## YouTube Sync {#youtube} -YouTube Sync has played a fundamental role in the previous months. YouTube has become more aggressive on censoring topics and channels on their platform and with their recent ban on crypto related videos we've observed a huge influx of new creators wanting to transfer to LBRY. During the first month and a half of the year we published more than 200,000 new videos with an average of 4500 videos every day! +YouTube Sync has played a fundamental role in the previous months. YouTube has become more aggressive in censoring topics and channels on their platform and with their recent ban on crypto related videos we've observed a huge influx of new creators wanting to transfer to LBRY. During the first month and a half of the year we published more than 200,000 new videos with an average of 4500 videos every day! -With so many new users and creators flooding into LBRY we had to better optimize the process of syncing new channels and keeping previous ones up to date. We have been very successful at that as we not only managed to keep up with the great demand, we also reduced the time it takes for popular channels to be updated: while in the past it would take up to 24 hours for a new video to appear on LBRY we've reduced this latency to up to 3 hours maximum. +With so many new users and creators flooding into LBRY we had to better optimize the process of syncing new channels and keeping previous ones up to date. We have been very successful at that as we not only managed to keep up with the great demand, we also reduced the time it takes for popular channels to be updated: while in the past it would take up to 24 hours for a new video to appear on LBRY, we've reduced this latency to up to 3 hours maximum. We're currently researching ways to further speed this process up as we'd like to see new videos come in as soon as they're released on YouTube. ![channels](https://spee.ch/8/sync-channels.jpeg) @@ -222,7 +222,7 @@ Finally, we’ve got [a big surprise scheduled for the final week of February](h ## 2019 review and new 2020 roadmap {#roadmap} We’re excited to announce our [2019 review by Jeremy](https://lbry.tv/@lbry:3f/LBRY-2019-Review:0) and the [roadmap for 2020](https://lbry.com/roadmap)! Jeremy covers the roadmap in details on our lbry.tv channel in both [video](https://lbry.tv/@lbry:3f/LBRY-2020-Roadmap-Release:6) and [text formats](https://lbry.tv/@lbry:3f/lbry-in-2019-2020:5). -Everyone at LBRY is excited about what’s in store for this year, especially with the interest we’ve seen in lbry.tv and more awesome creators coming on board. We hope to continue improving the creator experience through analytics and LBRY first publishing, which we think will change the game considerably (once content is exclusive on LBRY for the first X hours). +Everyone at LBRY is excited about what’s in store for this year, especially with the interest we’ve seen in lbry.tv and more awesome creators coming on board. We hope to continue improving the creator experience through analytics and LBRY first publishing, which we think will change the game considerably (once content is exclusive on LBRY for the first X hours). ## Limited Reward Code! We’ve got a special bonus for readers of this update, enjoy some LBC via this code (while supplies last!): `feb-dev-yuosd`