Are you independent and self-reliant? If the answer is ‘yes’, then we’re halfway there! Apply for a Lead Data Engineer position if you have:

• 5+ years of experience as a Data Engineer or in a similar role.
• Strong proficiency in Python (Mid/Senior level), R (Mid/Senior level), and experience with Java is a plus.
• Expertise in metadata cleanup and transformation, particularly for large datasets derived from web scraping processes.
• Hands-on experience with scaling and optimizing web scraping systems, including implementing best practices for data extraction and validation.
• Strong understanding of data normalization, deduplication, and enrichment techniques to ensure high-quality, reliable datasets.
• Familiarity with web scraping tools and frameworks such as Scrapy, BeautifulSoup, or Selenium, and ability to implement efficient scraping solutions.
• Knowledge of managing and updating product metadata structures in a dynamic commercial environment.
• Ability to troubleshoot and resolve issues related to inconsistent or incomplete data sources, ensuring seamless integration into existing data pipelines.
• Experience with NLP techniques, especially in prompt engineering for data automation.
• Proficient in designing and optimizing ETL pipelines and working with cloud data warehouses (e.g., Snowflake, BigQuery, Redshift).
• Familiarity with product metadata structures and experience working with large datasets in a commercial environment.
• Excellent problem-solving skills and attention to detail, especially in data validation and error handling.
• Strong communication skills to work effectively with cross-functional teams, including QA, backend developers, and product managers (English and Polish B2 + is a must).
• Experience with APIs, data integration, and handling multiple data sources.

Currently, we are looking for a person who will join the Grocery Buddy Team as a Lead Data Engineer. Your main responsibilities will include:

• Prompt Engineering & Scripts:
  • Create and optimize prompts/scripts to automatically clean, categorize, and tag product metadata.
  • Improve existing scripts for product types, sizes, tags, brands, and other metadata fields.

  Data Cleanup & Standardization:

  • Write scripts to handle data cleanup for brands, prices, sizes, and availability.
  • Convert size values to numerical representations based on product type-specific rules.

  Manual Review System:

  • Develop systems for manual review of flagged items in the data warehouse (DW).
  • Build review mechanisms for user-flagged issues and those identified by automated systems.

  Data Integration & Deduplication:

  • Ensure proper merging of products by identifying and consolidating duplicates using UPCs and metadata.

  Product Grouping:

  • Create scripts for identifying and grouping similar products based on size, color, scent, etc.

Grocery Buddy is a user-friendly mobile application designed to simplify everyday grocery shopping tasks. With its intuitive interface, users can efficiently plan their shopping lists and track their expenses in real-time. The app’s mission is to streamline the grocery shopping experience, offering convenience and organization to users’ daily routines.