60 points by databricks 2 years ago flag hide 13 comments
databrickshn 2 years ago next
We're Databricks (YC 2013) and we're actively hiring Data Engineers to join our growing team! Excited to share this news with the HN community. #hiring #dataengineering
friendly_dev 2 years ago next
Cool, congrats on the growth! Data Engineering has been an exciting field. I just recently started a Medium blog series on Apache Spark - hope it can help the community. #BigData #DataEngineering
learn_every_day 2 years ago next
friendy_dev - I love personal posting series! How about a tutorial on how to use Databricks (or even a comparison to other systems!). Would be so helpful for understanding your insights better ;)
dataninja 2 years ago prev next
Hiya Databricks, I'm a senior solutions engineer at one of your local competitors. I have a few questions regarding the tech stack you're using, may I PM you perhaps?
databrickshn 2 years ago next
@DataNinja - I would appreciate it if you don't mention who your employer is here on HN. We're more than happy to answer any questions about our platform in a generic context. Please message us directly for more information.
andra_rusu 2 years ago prev next
Interesting, I'm actually looking for a job. I have 4+ yrs as a data engineer with ML expertise. Working mainly on AWS stack at the moment. Would be thrilled to explore opportunities at Databricks. ps@..
empathetic_scientist 2 years ago prev next
I see that you're looking for experienced data engineers. What tools and technologies would I need experience in/get trained in before applying? Our team uses mostly cloud-based systems and we're looking to move our workload to lakehouses in the near future.
rubybyte 2 years ago next
Hey @empathetic_scientist, from the job postings I've seen - expertise in cloud-based systems like AWS, GCP or Azure along with knowledge of SQL and Python scripting is essential at Databricks. Working experience with Apache Spark, Delta Lake, and MLflow is considered a bonus.
sagar-gn 2 years ago prev next
Fascinating stuff happening at Databricks. I've been reading up on Delta Lake, a storage layer they open-sourced for Apache Spark that brings ACID transactions to big data workloads. Theoretically speaking, would such a toolset increase overall productivity in a #dataengineering role/team or perhaps cause additional pipeline bugs?
agile_geek 2 years ago prev next
@sagar-gn - from my experience with Databricks while landscaping the data streaming pipeline for a client - The implementation of tools like Delta Lake can provide a more stable platform for data engineers to build their pipelines at scale, but that depend on how early they embrace it. Gotta learn and try new stuff even it's hard. #BigData #DataPipeline
great_scott 2 years ago prev next
Don't want to hijack the thread, but I'd like to hear from other data engineers on this: What do you prefer - working with a data warehousing solution like Snowflake or Databricks? What are the benefits and disadvantages of each?
archerb 2 years ago next
As someone who has recently shifted from Snowflake to Databricks, I'm a fan of the adaptability that Delta Lake offers when things change. No more
archerb 2 years ago next
...long development cycles that I used to face with the rigid SQL and cryptic syntax. I'm now more productive and focused on solving business problems more efficiently. #BigData #DataLakehouse